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1.0  INTRODUCTION 


This  Final  Report  is  in  fulfillment  of  the  contractual  requirement  (CLIN  0001AD),  and 
provides  a  detailed  discussion  of  this  study  effort,  project  objectives,  work  performed, 
results  obtained,  and  estimates  of  technical  feasibility. 


2.0  BACKGROUND 

The  advent  of  the  head-up  display  (HUD)  has  afforded  improved  the  mission 
performance,  but  HUDs  suffer  from  several  limitations.  First,  the  fixed  nature  of  the 
HUD  restricts  the  pilot's  head  movement  to  the  front  of  the  aircraft.  While  the  HUD  has 
helped  to  allow  the  pilot  to  keep  the  head  up  and  “out  of  the  instrument  panel,”  the  pilot 
is  restricted  to  a  small  forward  looking  “design  eye”  for  viewing  information  displayed 
on  the  HUD.  Significant  head  movement  (e.g.,  scanning  for  other  aircraft,  ground  units, 
potential  foes,  etc.)  will  remove  the  pilots  point  of  view  out  of  the  “design  eye”  position 
and  render  the  HUD  useless.  The  forward-only  view  prevents  the  pilot  from 
simultaneously  scanning  the  aircraft  instruments  and  the  out-the-window  visual 
environment.  In  addition,  although  the  instantaneous  field-of-view  (FOV)  in  modem 
HUDs  is  approximately  25°,  HUDs  suffer  because  the  field-of-regard  (FOR)  is  nearly 
identical  to  the  FOV.  Anything  not  within  the  HUD  FOV  will  be  lost,  requiring  the  pilot 
to  maneuver  the  aircraft  so  that  the  object  of  interest  is  within  the  limits  of  the  HUD 
FOV. 

In  addition,  HUDs  require  a  great  deal  of  cockpit  “real  estate”  and  add  significant  weight 
to  the  aircraft.  Panel  space  is  already  limited  and  the  addition  of  new  weapons  systems 
will  only  serve  to  exacerbate  the  limited  panel  space  problem  in  the  future. 

Helmet-mounted  displays  (HMDs)  overlay  synthetic,  head-slaved  visual  information 
directly  over  the  view  of  the  out-of-the-cockpit  visual  environment.  These  systems 
afford  several  potential  advantages  for  pilots  of  tactical  aircraft  (Beal  &  Sweetman,  1994; 
Geiselman,  1999).  Unlike  conventional  “head-down”  instruments  that  force  the  pilot  to 
concentrate  his  attention  inside  the  cockpit,  or  Head-Up  Displays  (HUDs)  that  restrict  the 
pilot’s  attention  toward  the  front  of  the  aircraft,  HMDs  enable  direct  visual  access  to 
critical  information,  while  allowing  the  pilot  to  keep  his  line-of-sight  (LOS)  out  of  the 
cockpit,  uncoupled  with  the  longitudinal  axis  of  the  aircraft.  This  characteristic  of  HMD 
systems  can  offer  great  mission  flexibility  by  allowing  the  pilot  to  perform  mission- 
essential  tasks  requiring  off-boresight  LOS  operations  day  or  night,  in  all  weather 
conditions,  with  an  increased  margin  of  safety.  There  is  also  the  potential  for  increased 
safety  during  low  level,  terrain  following,  or  nap-of-the-earth  (NOE)  flight.  In  addition, 
HMD  systems  offer  significant  tactical  advantages  when  combined  with  helmet-mounted 
sight  (HMS)  weapons  systems  (Adam,  1994;  Beal  &  Sweetman,  1994)  by  allowing  pilots 
to  track  and  designate  high  off-boresight  targets  at  high  angles  off  the  nose  without  any 
concomitant  requirement  for  own-ship  maneuvering.  When  combined  with  high-off 
boresight  missile  capability  (Domheim  &  Hughes,  1995),  HMD/S  systems  allow  the  pilot 
a  greater  probability  of  achieving  a  “first  shot-first  kill.”  Finally,  in  addition  to  the 
operational  advantages  described  above,  the  potential  replacement  of  HUDs  by  HMDs 
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offers  significant  reductions  in  aircraft  weight  and  increased  availability  of  cockpit  “real 
estate,”  and  therefore  greater  system  design  flexibility  (Adam,  1994). 

The  current  emphasis  on  HMD  Research  and  Development  (R&D)  is  timely  in  that  the 
developmental  Joint  Strike  Fighter  (JSF)  is  planned  as  a  HUD-less  aircraft  equipped  with 
an  HMD  (Ashley,  1998).  A  team  led  by  Lockheed  Martin  won  the  Joint  Strike  Fighter 
competition  in  February  2002.  The  Lockheed  X-35  JSF  prototype  included  the  Vision 
Systems  International  (VSI)  Next  Generation  Helmet  (NGH)  advanced  HMD.  The  VSI 
HMD  is  an  integral  part  of  the  X-35  avionics  system  and  was  flight-tested  onboard  the 
Northrop  Grumman  Cooperative  Avionics  Testbed,  a  modified  BAC  1-11  aircraft 
(Lockheed  Martin,  2000). 

Much  of  the  recent  impetus  driving  the  US  commitment  to  HMD/S  technology  is  the 
perceived  threat  emanating  from  the  wide  export  market  of  Russian  and  Israeli  high-off- 
boresight  missiles  (AA-1 1  “Archer”  and  Rafael  Python  4,  respectively)  combined  with 
helmet-mounted  sights.  For  example,  the  AA-1 1  “Archer”  missile  with  helmet-mounted 
sight  targeting  is  being  offered  for  export  on  MiG-29  fighters,  and  as  an  upgrade  for  the 
older  MiG-21.  MiG-29  exports  alone  represent  approximately  750  aircraft  flown  by  17 
different  nations;  5500  MiG-2  Is  are  flown  by  40  nations  worldwide  (Domheim  & 
Hughes,  1995). 

The  Israeli  Display  and  Sight  System  (DASH)  HMD  and  sight  has  been  operational  for 
five  years,  is  currently  flown  by  four  air  forces,  and  is  operational  on  F-15,  F-16,  F-5,  and 
MiG-21  fighters.  The  potential  of  this  threat  was  evidenced  in  1994  during  exercises 
between  the  USAF  and  Germany  in  which  German  pilots  flying  MiG-29s  equipped  with 
Russian-built  AA-1 1  “Archer”  missiles  and  helmet-mounted  sights  achieved  the  first  shot 
against  US  Air  Force  F-16  fighters  in  60%  of  mock  engagements  (Hughes,  1995). 

Several  US  programs  have  engaged  in  investigation  of  the  tactical  implications  of 
HMD/S  systems  (Beal  &  Sweetman,  1994;  Domheim,  1995a;  Domheim,  1995b).  An 
early  HMS,  the  Honeywell  Visual  Target  Acquisition  System  (VTAS),  was  operational  in 
the  early  1970s  and  equipped  approximately  500  F-4  Phantom  fighters,  primarily  in  the 
US  Navy  and  Marine  Coips  (Domheim,  1995a;  1995b).  However,  limitations  in  the 
effectiveness  of  early  Sidewinder  missiles  and  the  advent  of  effective  close-range  off- 
boresight  automated  radar-lock  features  in  modem  digital  radars  prompted  the  removal  of 
VTAS  from  service.  Two  more  recent  programs  were  VISTA-  II  and  VISTA-  N.  These 
were  a  continuation  of  the  VISTA-  work  completed  during  the  1980s  designed  to  assess 
HMD/S  systems  and  to  delineate  deficiencies  with  current  HMDs  to  be  remedied  through 
requirements  for  future  designs.  The  VISTA-  II  and  VISTA-  N  program  plans  were  to 
equip  two  F-15Cs,  one  F-14A,  and  one  F/A-18C  with  HMD/S  systems  and  fly  numerous 
1  vs.  1,  and  2  vs.  2  aerial  engagements  for  evaluation. 

Another  US  HMD  project  was  the  Visually  Coupled  Acquisition  and  Targeting  System 
(VC ATS).  This  program  was  designed  to  achieve  a  producible  HMD/S  system  that 
satisfied  the  requirements  developed  in  the  VISTA-  II  program.  The  VCATS  program 
was  scheduled  to  terminate  with  the  first  flight  of  an  F-15C  equipped  with  the  VCATS 
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system.  The  results  of  the  VCATS  Technology  Demonstration  program  fed  into  the  Joint 
Helmet  Mounted  Cueing  System  (JHMCS).  The  JHMCS  is  the  executor  of  a  recently 
completed  Operational  Requirements  Document  with  the  goal  of  deploying  HMD/S 
systems  into  operational  cockpits  beginning  around  2001  (Domheim  &  Hughes,  1995; 
Domheim,  1995a). 

VSI,  Inc.  won  a  contract  to  produce  the  JHMCS  HMD  in  1997.  The  JHMCS  helmet  was 
the  subject  of  recent  flight  testing  conducted  at  Edwards  Air  Force  Base  by  the  F-15 
Combined  Test  Force,  with  the  first  flight  on  October  22, 1998  (McQuillan,  1999).  The 
flight-testing  plan  consisted  of  70  missions  and  included  developmental  testing  of 
JHMCS  through  tests  combining  the  JHMCS  HMD  with  the  AIM-9X  off-boresight 
missile  system. 

The  recent  activity  centered  on  HMD-oriented  research  has  created  a  number  of 
important  R&D  issues  that  must  be  solved  in  order  to  optimize  performance  throughout 
all  mission  areas  for  these  systems  to  be  successful.  Among  the  primary  research  issues 
to  be  addressed  in  the  development  of  HMDs  are  those  related  to  determining:  (1)  the 
amount  and  type  of  information  to  be  displayed,  (2)  the  most  effective  presentation,  and 
(3)  methods  for  optimizing  the  integration  of  information  presented  via  the  HMD  with 
other  visual  and  non-visual  displays.  In  order  to  address  these  questions,  NTI  proposed 
the  use  of  an  R&D  methodology  that  has  been  successfully  applied  to  design  and  evaluate 
advanced  tactical  crew  station  concepts  including  HMDs  (Brickman,  Hettinger,  &  Haas, 
2000;  Brickman,  Hettinger,  &  Haas,  1997;  Geiselman,  Brickman,  Hettinger,  Hughes, 
DeVilbiss,  &  Haas,  1998;  Nelson,  Hettinger,  Cunningham,  Brickman,  Haas,  & 

McKinley,  1998).  This  methodology  centers  on  the  use  of  an  iterative  multidisciplinary 
design  team  approach  discussed  below. 


3.0  PROGRAM  OBJECTIVES 

3.1  General 

The  principal  technical  objectives  of  this  Phase  I  program  were  as  follows: 

•  Define  the  human  performance  requirements  supported  by  current  aircraft-stabilized 
HUD  designs  as  a  basis  for  assuring  that  those  requirements  are  satisfactorily  met  in 
any  proposed  HMD  design. 

•  Define  the  human  performance  requirements  for  a  proposed  HMD  system  designed  to 
replace  current  HUD  information  support  capabilities  while  also  incorporating  off- 
boresight  information  capabilities.  These  information  support  capabilities  will 
address  air-to-air,  air-to-ground,  and  flight  navigation  requirements. 

•  Deliver  a  detailed  functional  specification  of  a  candidate  HMD  symbology  set 
meeting  the  requirements  defined  in  the  objective  above  (including  symbology 
format,  functionality,  and  mechanization)  capable  of  providing  current  HUD-based 
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information  while  also  incorporating  current  and  novel  display  techniques  to  support 
pilot  performance  in  air-to-air,  air-to-ground,  and  flight  navigation  tasks. 


•  Provide  a  detailed  Technical  Report  to  describe  our  work,  findings,  and  functional 
specifications  of  the  candidate  designs  developed. 


3.2  Discussion 

The  major  objective  of  this  Phase  I  effort  was  to  develop  and  deliver  to  the  Government  a 
detailed  functional  specification  of  a  candidate  HMD  symbology  set.  By  “detailed 
functional  specification”  we  mean  a  written  and  graphical  description  of  the  HMD 
symbology  format,  functionality,  and  mechanization  at  a  level  of  detail  sufficient  to 
permit  prototype  development  to  occur  immediately  at  the  outset  of  a  Phase  II  Small 
Business  Innovation  Research  (SBIR)  program,  or  a  similar  research  effort.  This 
functional  specification  should  permit  limited  demonstration  of  the  concept’s 
functionality.  Its  primary  purpose  will  be  to  permit  full-scale  prototype  development  and 
testing  to  occur  in  follow-on  research  efforts. 

In  this  report  the  term  HMD  symbology  set”  refers  to  the  broad  range  of  interface 
capabilities  and  characteristics  implied  by  the  terms  “HMD  symbology  format, 
functionality,  and  mechanization.”  These  terms  have  the  following  meanings: 

•  Symbology  Format  -  The  term  “symbology  format”  refers  to  the  description  of  what 
is  actually  displayed  on  the  HMD.  This  includes  the  symbols  and  colors  used,  as  well 
as  techniques  employed  for  displaying  information,  such  as  digital  versus  analog, 
“round  dials”  versus  “tapes,”  “  pitch  ladders”  versus  “orange  peels,”  “inside-out” 
versus  “outside-in”  viewpoints,  and  distributed  versus  non -distributed  formats. 

•  Symbology  Functionality  -  This  term  refers  to  why  the  information  is  to  be  displayed, 
that  is,  the  purpose  of  the  information.  The  objective  of  this  development  was  to 
provide  all  the  functionality  of  current  HUDs,  while  extending  their  utility  to  high 
off-boresight  angles.  This  functionality  includes  primary  flight  reference  (attitude, 
airspeed,  altitude,  heading,  etc.),  navigation  support,  sensor  targeting,  and  weapon 
system  status.  The  challenge  of  this  effort  was  to  retain  and  extend  this  functionality 
to  the  high  off-boresight  regime  in  an  optimal  manner. 

•  Symbology  Mechanization  -  This  term  refers  to  the  “behavior”  of  the  symbology  in 
dynamic  situations.  In  other  words,  “symbology  mechanization”  refers  to  how  the 
symbology  behaves  (symbol  color  changes,  changes  in  size/shape,  movement,  etc.)  in 
response  to  events  occurring  in  the  tactical  and  in-flight  environments.  Another 
important  issue  in  off-boresight  mechanization  is  how  the  information  is  referenced, 
e.g.,  to  the  aircraft,  to  the  head,  to  the  eye,  to  some  combination,  etc. 

The  NTI  Team  recognized  the  potential  of  the  HMD  as  a  platform  for  the  delivery  of 
multi-sensory  information.  As  we  interpreted  the  requirements  of  this  SBIR,  however, 
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the  intent  was  to  concentrate  on  the  VISUAL  mode  of  presentation.  Recommendations 
and  guidelines  are  also  provided,  however,  regarding  additional  sensory  modes,  such  as 
spatial/3-D  audio. 


4.0  STUDY  PLAN 

The  NTI  Team  addressed  the  tasks  above  by  application  of  the  principles  and  techniques 
of  User-Centered  Design  that  we  had  developed  and  applied  in  similar  applications. 

These  include  support  of  the  USAF’s  Synthesized  Immersion  Research  Environment 
(SIRE)  Supercockpit  program,  and  more  recently  in  support  of  the  US  Navy’s  DD-21 
program.  Our  approach  to  the  design  of  innovative  pilot-vehicle  interfaces  is  described  in 
Brickman  et.  ah,  (1998),  and  in  the  remainder  of  this  section.  In  addition,  a  number  of 
publications  and  technical  reports  are  available  that  describe  applications  of  our 
methodology  in  the  design  of  such  interfaces  (e.g.,  Geiselman  et.  ah,  1998;  Haas, 
Hettinger,  Nelson,  &  Shaw,  1995;  Hettinger,  Tannen,  Geiselman,  Brickman,  Moroney, 
and  Haas,  1998;  Hettinger,  Nelson,  &  Haas,  1994;  1996). 


4.1  Cognitive  Work  Analysis  (CWA) 

A  key  part  of  our  design  effort  involved  the  performance  of  Cognitive  Work  Analyses. 
The  purpose  of  these  analyses  was  to  provide  a  structured  setting  within  which  Subject 
Matter  Experts  (SMEs)  could  collectively  analyze  representative  mission  scenarios, 
discuss  the  deficiencies  of  current  HUD  information  formats  and  functionality,  and 
propose  and  discuss  new  HMD-based  design  concepts.  The  outcome  of  these  analyses 
was  the  prototype  design  concepts  presented  and  discussed  elsewhere  in  this  report. 

CWA  is  a  technique  that  has  become  increasingly  widespread  in  multi-disciplinary 
design  team  settings  (Rasmussen,  Pejtersen,  &  Goodstein,  1994;  Vicente,  1999).  While 
CWA  techniques  are  applicable  to  a  wide  variety  of  human-machine  system  issues,  such 
as  the  analysis  of  unsafe  conditions  associated  with  existing  technologies,  we  have  found 
them  to  be  very  useful  as  a  tool  for  focusing  user-centered  design  teams  in  aviation 
settings  (e.g.,  Brickman,  et.  al.,  2000;  Brickman,  Hettinger,  Haas  &  Dennis,  1998)  and 
medical  settings  (Hettinger  et.  al.,  1998). 

As  Naikar,  Lintem,  and  Sanderson  (2001)  state,  CWA  consists  of  the  following  five 
components: 

•  Work  Domain  Analysis  -  an  examination  of  the  purpose  of  the  system  under 
consideration  as  well  as  its  physical  functions,  including  analysis  of  the  situations 
within  which  the  system  operates.  In  our  case,  this  included  an  analysis  of  existing 
HUD  and  HMD  display  concepts  and  functionality,  as  well  as  an  examination  of  a 
specific  set  of  tactical  scenarios  (i.e.,  air-to-ground,  night  strike,  SCUD-hunting 
missions)  in  which  the  new  HMD  system  would  be  expected  to  operate.  For  design 
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purposes.  Work  Domain  Analysis  helped  to  define  the  functional  “boundaries”  within 
which  an  HMD  interface  must  successfully  support  pilot-aircraft  system  performance. 

•  Control  Task  Analysis  -  an  examination  of  the  activities  that  must  be  performed  in 
order  for  a  system  to  meet  its  performance  objectives.  In  our  analysis,  this  took  the 
form  of  identifying  and  discussing  effective  pilot-aircraft  system  behaviors/tactics 
that  needed  to  be  successfully  executed  for  each  of  the  scenarios  that  were  examined. 
For  design  purposes,  Control  Task  Analysis  provided  information  on  key  aspects  of 
system  behavior  that  must  be  supported  by  an  HMD  interface. 

•  Strategies  Analysis  -  an  examination  of  the  ways  that  tasks  and  activities  are  carried 
out.  In  our  analysis,  this  took  the  form  of  individual  SMEs  elaborating  on  possible 
methods  that  could  be  used  to  accomplish  various  mission  and  intra-mission 
objectives  involved  in  the  targeted  scenarios.  For  design  purposes,  Strategies 
Analysis  enabled  the  design  team  to  appreciate  the  nature  of  individual  differences  in 
task  performance,  and  how  interface  design  would  need  to  be  able  to  accommodate 
differences  in  individual  strategies  and  techniques. 

•  Socio-Ormnizational  A nalysis  -  an  examination  of  how  work  is  shared  among 
members  of  a  work  team.  In  our  case,  this  took  the  form  of  examining  how 
coordinated  groups  of  fighter  aircraft  approach  the  execution  of  tactical  scenarios. 
Specifically,  we  examined  the  importance  of  such  factors  as  handing  off 
responsibility  for  targets,  the  function  of  datalink  capabilities,  and  other  coordinated, 
multi -aircraft  tactical  activities.  For  design  purposes,  Socio-Organizational  Analysis 
provided  the  team  with  knowledge  of  an  additional  critical  set  of  constraints  within 
which  the  HMD  interface  would  need  to  operate. 

•  Worker  Competencies  Analysis  -  an  examination  of  the  training,  knowledge,  skills, 
and  abilities  that  individuals  must  possess  in  order  to  carry  out  the  types  of  tasks 
being  analyzed,  including  analysis  of  the  different  skill  levels  possessed  by 
individuals  with  different  training  and  experience  backgrounds.  In  our  analysis  this 
took  the  form  of  examining  the  importance  of  such  factors  as  overall  flight 
experience,  experience  in  different  types  of  tactical  scenarios,  experience  in  different 
types  of  aircraft,  etc.  on  the  performance  of  the  tasks  under  consideration.  For 
purposes  of  design  this  information  is  critical,  because  the  HMD  interface  under 
consideration  is  intended  to  support  effective  performance  for  operators  with  widely 
variable  backgrounds. 

For  this  effort,  CWA  activities  concentrated  on  the  extraction  of  subject  matter  expertise 
directly  relevant  to  the  design  of  a  novel  HMD  concept  intended  to  be  functionally 
superior  to  existing  HUD  displays.  No  attempt  was  made  to  prepare  an  abstraction 
hierarchy  or  other  intermediate  products  frequently  associated  with  CWA  techniques. 
Instead,  the  discussion  of  CWA  topics  listed  above  was  intended  to  result  in  the 
immediate  introduction  of  design  suggestions  by  the  SMEs.  These  suggestions  were 
captured  by  design  team  personnel  on  paper  for  later  mock-up  on  a  personal  computer 
platform.  In  addition,  the  room  in  which  the  CWA  took  place  included  a  white  board, 
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colored  markers,  and  large  sheets  of  paper  which  were  used  to  depict  design  concepts, 
and  to  modify  them  in  response  to  group  suggestions.  Therefore,  the  CWA  questions  and 
discussions  served  primarily  as  a  vehicle  for  eliciting  design  concepts  from  the  SMEs. 


4.2  DEFINING  HUD  REQUIREMENTS 

The  first  objective  of  this  Phase  I  SBIR  program  was  to  define  the  human  performance 
requirements  supported  by  current  aircraft-stabilized  HUD  designs  as  a  basis  for  assuring 
that  those  requirements  are  satisfactorily  met  in  any  proposed  HMD  design.  Once 
defined,  HUD  design  parameters  became  the  starting  point  for  HMD  design,  and  a  basis 
against  which  HMD  requirements  could  be  compared  to  ensure  completeness. 

The  performance  requirements  imposed  on  tactical  aviation  pilots  are  dictated  by  the 
demands  of  any  particular  mission.  Within  a  given  mission,  the  human  performance 
requirements  are  largely  identical  regardless  of  whether  the  aircraft  has  no  HUD,  is  HUD 
equipped,  or  is  equipped  with  an  HMD  (in  the  sense  that  every  mission  requires  that  the 
pilot  successfully  aviate,  navigate,  and  communicate).  All  else  being  equal,  the  aircraft 
that  offers  the  pilot  comprehensive  coverage  of  the  information  requirements  associated 
with  a  particular  mission  in  the  most  comprehendible  format  will  enjoy  better  mission 
performance.  However,  there  are  some  key  differences  in  capability  that  exist  between 
HUD-equipped  and  HMD-equipped  aircraft  that  should  afford  enhanced  mission 
capability  for  the  HMD  equipped  fighter. 

Since  HUDs  have  been  common  in  military  aircraft  since  the  1970s,  accepted 
informational  requirements  are  fairly  well  established.  The  HCDT  began  its  development 
of  HMD  informational  requirements  using  the  HUD  as  an  initial  baseline.  The  primary 
reference  employed  to  identify  HUD  informational  requirements  for  this  purpose  was 
Newman  (1995),  HUDs:  Designing  the  way  ahead.  This  reference,  as  well  as  the  Pi’s 
tactical  experience  flying  HUD-equipped  fighters  provided  the  initial  set  of  HMD 
informational  requirements,  as  described  below. 

The  primary  benefits  of  the  HUD  include  reduced  pilot  workload  when  tasks  require 
head-up  flight,  increased  flying  precision,  direct  visualization  of  the  aircraft  trajectory, 
and  increased  safety  of  flight  by  allowing  the  pilot  to  “keep  the  eyes  out  of  the  cockpit” 
(Newman,  1995). 

Any  HUD  symbology  set  should  include  elements  that  aid  the  pilot  in  performing  basic 
tasks  associated  with  aviating,  navigating,  and  communicating.  In  order  to  support  these 
behavioral  goals,  minimum  HUD  symbology  sets  have  typically  included  attitude,  pitch 
ladder,  waterline,  current  altitude,  vertical  velocity,  airspeed  (including  MACH  if 
operationally  necessary),  and  heading  symbology  (Newman,  1995).  These  elements 
comprise  the  basic  aircraft  parameters  required  to  fly  and  navigate  that  are  usually  found 
arranged  in  the  “standard  T”  layout  on  the  cockpit  instrument  panel.  Further,  most 
typical  HUD  symbology  sets  have  included  a  Climb/Dive  Marker  (CDM)  or  Flight  Path 
Marker  (FPM)  added  to  the  basic  symbology  set.  Finally,  indications  of  aircraft  energy, 
radar  altitude  (AGL),  and  additional  altitude  displays  have  also  provided  useful  additional 
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information  in  HUD  designs.  Flight  director  and  lateral  deviation  indicators  have  also 
been  added  to  the  HUD  symbology  set,  dependent  upon  the  navigation  requirements  of 
the  aircraft. 

For  air  combat  applications,  the  typical  information  set  has  been  enhanced  with  the 
addition  of  aiming  symbology,  aircraft  load  factor,  and  information  about  the  target 
(distance,  range).  The  precise  “aiming  symbology”  parameters  that  are  included  in  the 
HUD  symbology  set  are  typically  specific  to  the  weapon  system  in  question.  Specific 
weapon  system  application  examples  were  not  included  in  Newman  (1995),  however,  a 
number  of  general  weapon  system  information  parameters  have  been  included  in  the 
HMD  symbology  set.  For  HUD  Air-to-Ground  applications,  true  airspeed,  bank  scale, 
and  breakaway  distance  displays  have  been  commonly  added  to  HUD  symbology. 

5.0  PROGRAM  CHRONOLOGY 
5.1  KICK-OFF  MEETING 

The  contract  for  this  Phase  I SBIR  program  began  officially  on  10  May  2001.  The 
program  Kick-Off  Meeting  was  conducted  at  Wright-Patterson  AFB  (WPAFB),  OH,  on 
24  May  2001.  This  meeting,  conducted  by  Mr.  Robert  Shaw,  NTI  Principal  Investigator 
(PI),  and  assisted  by  Mr.  Mark  Crabtree  (NTI),  was  well  attended  by  a  number  of 
interested  Government  parties,  including  Mr.  Eric  Geiselman,  2Lt  Chris  Jenkins 
(Government  Program  Manager),  Dr.  Paul  Havig  (Northrop-Grumman/Logicon),  and 
others.  Also  present  was  Dr.  Jennie  Gallimore  (NTI  consultant),  while  Mr.  Bart 
Brickman  (Northrop-Grumman/Logicon)  attended  via  teleconference. 

Mr.  Shaw  provided  an  overview  of  the  program,  including  the  objectives,  the 
NTI/Logicon  research  team,  the  Work  Plan,  and  the  schedule.  A  useful  discussion 
followed  the  presentation.  The  Government  Program  Manager  suggested  de-emphasizing 
the  proposed  evaluation,  which  should  allow  more  effort  to  be  directed  toward  “fine- 
tuning”  the  specifications  for  the  HMD  symbology.  It  was  stated  that  the  target  platforms 
for  the  HMD  symbology  to  be  developed  during  this  program  are  the  F-15E  Strike  Eagle 
and  the  Strike  Helmet  21,  which  establishes  the  HMD  technology  base  to  be  assumed, 
including  color  imagery,  direct  voice  interaction  (DVI),  and  possibly  3-D  audio 
capability.  The  primary  mission  of  interest  was  stated  as  the  night  air-to-ground  scenario, 
both  in  and  out  of  adverse  weather.  It  was  also  offered  that  Maj  Mike  Brewer  (F-22 
SPO)  might  be  a  good  candidate  for  the  Human-Centered  Design  Team  (HCDT),  and 
could  also  be  helpful  in  providing  SMEs  to  support  the  program. 


5.2  LITERATURE  SEARCH 

A  supporting  literature  search  was  begun  immediately  on  contract  award  and  continued 
throughout  the  program.  A  large  number  of  relevant  references  were  identified  and 
obtained.  Lists  of  these  references  were  distributed  to  the  HCDT  for  information,  and  to 
solicit  additional  material.  A  final  compendium  of  these  references  is  provided  in  the 
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Selected  References  section  of  this  report.  So  as  not  to  “reinvent  the  wheel,”  the  results 
of  prior  relevant  studies  were  incorporated  into  our  design  decisions  throughout  the 
program. 


5.3  FORMATION  OF  THE  HUMAN-CENTERED  DESIGN  TEAM 

Following  the  Kick-Off  Meeting,  the  HCDT  was  formed  as  follows: 

>  Robert  Shaw  (PI) 

>  2Lt  Chris  Jenkins  (AFRL/Govemment  Program  Manager) 

>  Eric  Gieselman  (AFRL) 

>  Dr.  Larry  Hettinger  (Logicon) 

>  Bart  Brickman  (Logicon) 

>  Mark  Crabrtee  (NTI) 

>  Maj  Mike  Brewer  (F-22  SPO) 

>  3-4  SMEs  (TBA) 

The  SMEs  were  not  identified  at  this  time,  but  those  who  eventually  participated,  and 
their  qualifications,  are  provided  later  in  this  report.  The  purpose  of  the  HCDT  was  to 
work  closely  together  toward  the  objectives  of  this  program.  This  might  include  actual 
participation  in  CWA  sessions,  reviewing  draft  documents  generated,  or  simply  acting  in 
an  advisory  capacity. 

This  team,  plus  other  interested  Government  parties  and  Dr.  Paul  Havig  (Logicon),  minus 
the  SMEs,  attended  the  initial  HCDT  Technical  Meeting  at  WPAFB  on  15  June  2001. 

Mr.  Shaw  provided  a  brief  refresher  of  the  objectives  of  the  program  and  the  guidelines 
established  thus  far.  The  discussion  then  centered  on  the  lack  of  availability  of 
supporting  SMEs  for  this  program.  Maj  Brewer  suggested  three  possibilities: 

•  422  Test  &  Evaluation  Squadron  (422  TES)  at  Nellis  AFB  NV 

•  USAF  Test  Pilot  School  (TPS)  at  Edwards  AFB  CA 

•  Operational  F-15E  squadrons,  most  likely  at  Seymour-Johnson  AFB  NC 

An  immediate  inquiry  with  AFRL/HE  management  eliminated  the  422  TES  as  being  too 
heavily  tasked  to  be  able  to  participate.  Likewise,  the  proposed  TPS  student  project 
participation  would  need  to  occur  in  the  January  through  March  timeframe,  which  was 
beyond  the  target  completion  date  of  this  program.  An  action  item  was  generated  for 
Maj.  Brewer  to  continue  attempts  to  arrange  SME  support  from  Seymour-Johnson.  The 
optimal  level  of  support  was  established  at  3-4  SMEs  for  3-4  sessions  of  about  four  hours 
each. 

An  additional  action  item  was  generated  for  Mr.  Shaw,  based  on  his  substantial 
experience  in  fighter  aviation  with  both  the  USAF  and  U.S.  Navy,  to  take  a  “first  cut”  at 
generating  a  draft  mission  description  and  scenario  for  use  in  generating  HMD 
symbology  requirements  and  concepts,  and  for  a  basis  of  discussion  with  SMEs.  As  an 
initial  step,  a  high-level  mission  breakdown  was  developed  and  distributed  to  HCDT 
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members  for  comment.  This  breakdown  listed  Mission  Phases  and  likely  weapons, 
sensors,  and  support  facilities  available  for  the  three  primary  F-15E  Air-to-Ground 
missions: 

•  Air  Interdiction  (fixed  targets) 

•  Close  Air  Support 

•  Mobile  Pop-Up  Target /SCUD  Hunt 

Further  discussions  with  Government  HCDT  members  established  that  the  Mobile  Pop- 
Up  Target/SCUD  Hunt  mission  was  of  primary  concern  and  should  be  the  emphasis  of 
this  Phase  I  development  effort.  Therefore,  a  detailed  description  of  the  requirements  of 
this  mission,  as  related  to  HMD  issues,  was  developed,  as  described  below. 


5.4  MISSION  DESCRIPTION  /  HMD  REQUIREMENTS  DEFINITION 

The  PI  generated  a  draft  document  entitled,  ANALYSIS  OF  HMD  SYMBOLOGY 
REQUIREMENTS  FOR  THE  F-15E  NIGHT  MOBILE-TARGET /  SCUD-HUNT 
MISSION  (Appendix  A),  for  use  as  a  guide  to  further  efforts  in  this  development 
program.  This  selected  mission  was  deconstructed  into  the  following  operational  mission 
phases: 

>  Ingress 

>  Target  Search  /  Detection  /  Identification 
y  Weapon  Delivery 

>  Egress 

For  each  of  these  mission  phases,  the  associated  pilot  tasks  were  described  and  issues 
possibly  related  to  HMD  design  were  discussed.  Based  on  this  detailed  analysis,  an 
initial  list  of  required  HMD  functionalities  and  requirements  was  also  generated.  The 
draft  mission  description  and  HMD  functional  requirements  list  were  reviewed  by  other 
members  of  the  HCDT.  Maj  Brewer  was  particularly  helpful  in  this  review  due  to  his 
extensive  operational  fighter  experience.  Comments,  revisions,  and  suggestions  were 
incorporated  into  both  documents.  The  draft  HMD  Requirements  List  became  the 
roadmap  for  discussions  during  subsequent  sessions  with  our  pilot  SMEs,  and  was 
revised  after  each  of  these  sessions  based  on  SME  inputs.  The  final  version  of  the  HMOD 
Requirements  List  is  provided  in  APPENDIX  C. 


5.5  SUBJECT  MATTER  EXPERT  SUPPORT 

During  August,  Maj.  Brewer  was  able  to  arrange  the  support  of  an  F-15E  squadron  at 
Seymour- Johnson  AFB  NC.  Phone  and  e-mail  discussions  with  the  squadron  Operations 
Officer,  LTC  Pat  Pence,  indicated  that  he  and  his  squadron  would  be  happy  to  support 
our  program  in  any  way  possible.  Consequently,  we  requested  the  availability  of  3-4 
pilot  SMEs,  for  3-4  hours  a  day,  for  3-4  days  during  our  first  visit,  tentatively  arranged 
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for  the  week  of  17-21  September.  Unfortunately,  during  the  first  week  of  September 
LTC  Pence  informed  the  PI  that  higher  authorities  had  gotten  involved  and  decided  that 
the  F-15E  wing  should  not  support  our  program  without  direction.  Instead,  they 
suggested  we  contact  the  422  Test  &  Evaluation  Squadron  (422  TES)  at  Nellis  AFB,  NV, 
for  SME  support.  Of  course,  as  discussed  earlier,  we  had  already  considered  and 
eliminated  this  option  based  on  AFRL/HE  direction. 

Consequently,  we  again  began  the  process  of  identifying  qualified  SMEs  in  the  local 
Dayton  area.  This  false  start,  however,  put  our  program  well  behind  schedule. 
Fortunately,  we  were  eventually  able  to  obtain  the  support  of  an  excellent  group  of  pilot 
SMEs.  SME  names  and  experience  are  provided  in  TABLE  1,  listed  in  alphabetical 
order.  In  addition,  to  these  SMEs  the  PI  and  Maj.  Brewer  continued  to  apply  their 
respective  fighter-pilot  backgrounds  as  members  of  the  HCDT. 


TABLE  1.  Characteristics  of  Subject  Matter  Experts  Who  Participated  in 
Cognitive  Work  Analysis  Sessions. 


Pilot 

Current 

Status 

Flight 

Hours 

Military 

Aircraft  Flown 

F-22  SPO 

MBim 

F- 1 5 A/B/C/D/E,  T/AT-38 

LTC  Mike  Green 

OHANG 

F-16  pilot 

2,900 

F-16C/D,  F-4C/D/E, 
AT-38B,  T-38A,  T-37B 

Maj.  Jeff  Lay 

OHANG 

F-16  pilot 

3,000 

F-16C/D,  F-14A/B/D, 
F-18D,  T-34C,  T-2C,  TA4J 

Bill  Simmons 

Retired  OHANG 
A-7  pilot 

5,400 

A-7D/K,  T-38,  F-4,T-37, 
F-5,  T-33,  F-100 

Maj.  Cecil  Stine 

OHANG 

F-16  pilot 

3,000+ 

F-16C/D,  F-15A/B/C/D, 
T/AT-38,  T-37 

5.6  COGNITIVE  WORK  ANALYSIS  SESSIONS 
5.6.1  CWA  SESSION  1 

We  conducted  three  separate  CWA  sessions.  The  first  session  was  conducted  over  two 
days  on  17-18  October  2001.  All  five  experienced  fighter-pilot  SMEs  listed  in  TABLE  1 
above  participated  for  four  hours  on  the  first  day,  but  one  SME  was  unavailable  for  the 
similar  session  on  the  following  day.  The  PI  moderated  the  sessions  with  the 
collaboration  of  our  Human  Factors  specialists,  Dr.  Larry  Hettinger  and  Bart  Brickman  of 
Northrop-Grumman/Logicon.  An  audio  recording  was  generated  of  both  these  sessions 
for  later  reference.  Mr.  Brickman  and  the  PI  continued  the  CWA  process  during  the 
morning  of  the  third  day  in  order  to  summarize  and  document  the  data  generated  during 
the  SME  sessions. 
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Prior  to  the  initial  session,  all  five  SMEs  were  provided  with  the  document  entitled 
Analysis  of  HMD  Symbology  Requirements  for  the  F-15E  Night  Mobile-Target/SCUD- 
Hunt  Mission  (Appendix  A),  described  previously,  in  order  to  provide  the  SMEs  with  an 
introduction  to  the  tactical  context  within  which  the  CWA  discussions  would  occur. 

At  the  beginning  of  the  first  session,  the  SMEs  were  also  provided  another  document  that 
listed  the  general  questions  and  topics  that  would  serve  as  the  focus  of  discussion  (see 
Appendix  B).  Following  discussion  and  review  of  the  F-15E  Night  Mobile-Target  / 
SCUD-Hunt  mission,  the  design  group  and  SMEs  embarked  on  a  discussion  on  all  the 
issues  and  questions  contained  in  Attachment  2.  In  doing  so,  the  design  group 
continually  elicited  design  recommendations  for  a  novel  HMD  system,  and  a  process  of 
sketching  out  and  modifying  candidate  designs  was  initiated. 

The  outputs  of  the  first  CWA  session  were:  (1)  preliminary  HMD  design  concepts,  and 
(2)  a  preliminary  list  of  HMD  symbology  requirements  (see  APPENDIX  C).  The  latter 
provides  information  regarding  the  SMEs’  consensus  on  how  specific  data  elements  (e.g., 
magnetic  heading,  magnetic  course,  attitude,  etc.)  should  be  presented  within  an  HMD  ° 
context.  Of  particular  concern  was  whether  or  not  individual  data  elements  should  be 
stabilized  with  respect  to  the  earth,  aircraft,  or  head. 

During  these  discussions  the  SMEs  stressed  the  importance  of  reducing  clutter  in  the 
HMD,  and  expressed  the  need  for  considerable  “declutter”  capabilities.  The  extensive 
use  of  color  was  noted  as  highly  desirable  for  an  HMD  to  enhance  situation  awareness 
(SA)  through  improved  intuitiveness,  reduced  workload,  and  reduced  clutter.  The  SMEs 
also  embraced  the  concept  of  a  Virtual  HUD”  (VHXJD).  This  concept  provides  for  the 
display  of  a  HUD-like  area  positioned  in  the  pilot’s  FOV  when  looking  forward  along 
the  longitudinal  axis  of  the  aircraft.  This  viewing  region  would  only  be  available  to  the 
pilot  when  his  LOS  was  forward,  as  with  a  conventional  HUD,  and  would  be  stabilized 
relative  to  the  aircraft,  but  displayed  on  the  HMD.  The  VHUD  would  typically  contain 
information  not  ordinarily  considered  critical  by  the  pilot  except  when  looking  forward, 
or  information  that  the  pilot  would  be  willing  and  able  to  look  forward,  at  least 
momentarily,  in  order  to  view.  Some  data  elements  always  available  on  the  HMD  would 
also  be  repeated  on  the  VHUD,  but  might  be  provided  in  a  different,  more  detailed  format 
to  enable  more  precise  aircraft  control  for  tasks  requiring  such  precision. 

It  was  widely  agreed  that  precise  control  was  not  a  critical  task  while  the  pilot  was 
looking  at  large  off-boresight  angles  from  the  longitudinal  axis  of  the  aircraft.  Targeting 
and  spatial  orientation  were  viewed  as  much  more  critical  functions  of  the  HMD  at  large 
off-boresight  viewing  angles.  The  support  of  pilot  spatial  awareness  typically  requires, 
and  is  often  actually  enhanced  by,  reduced  precision  (and  presumably  reduced  clutter)  in 
flight-data  displays.  For  more  precise  control,  such  as  might  be  desirable  for  instrument 
approaches,  landing,  low-level  flight,  etc.,  the  pilot  is  willing  (and  prefers)  to  restrict  his 
LOS  to  the  front  of  the  aircraft.  Similarly,  the  SMEs  drew  a  distinction  between  the 
functions  of  spatial  orientation  and  “unusual-attitude  recovery.”  Obviously,  enhancing 
the  former  should  reduce  the  necessity  of  the  latter  function.  In  addition,  the  SMEs  were 
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in  agreement  that  they  would  prefer  to  perform  unusual-attitude  recoveries,  once  the 
situation  was  first  identified  or  suspected,  while  looking  forward. 

During  this  session  the  SMEs  were  also  introduced  to  a  proposed  Ground  Collision 
Avoidance  System  (GCAS)  display  concept,  designed  to  provide  the  pilot  with  warning 
and  guidance  for  a  timely  recovery  from  potentially  critical  low-altitude  situations.  This 
concept  will  be  described  in  greater  detail  later  in  this  report.  The  SMEs  heartily 
embraced  this  concept.  They  were  also  typically  positive  to  the  concept  of  an  automated 
GCAS,  as  long  as  they  had  the  option  of  disabling  it. 


5.6.2  CWA  SESSION  2 

In  preparation  for  the  second  CWA  session,  thirty  (30)  PowerPoint  slides  were  developed 
to  illustrate  head-referenced  symbology  (visible  wherever  the  pilot  looks),  earth- 
referenced  symbology  (visible  only  when  the  pilot  is  actually  looking  along  the  object's 
line-of-sight),  and  an  aircraft-referenced  VHUD.  Particular  attention  was  paid  to  how 
symbology  having  these  various  reference  systems  would  interact. 

Our  second  CWA  session  was  conducted  on  18  December  2001,  headed  by  the  PI.  Pilot 
SMEs  participating  were:  LTC  Mike  Green,  LTC  Bill  Simmons,  and  LTC  Dan  Draeger. 
Two  of  the  original  SME  team  (Maj  Lay  and  LTC  Stine)  were  unable  to  attend  due  to 
priority  commitments.  As  during  the  first  CWA  session,  all  discussions  were  recorded  on 
audio  tape. 

The  session  began  with  a  review  of  the  HMD  Symbology  Requirements  (APPENDIX  C) 
document  generated  during  the  first  CWA  session  in  order  to  clarify  several  points  and  to 
ensure  proper  interpretation  of  SME  suggestions  provided  during  that  earlier  session. 

Only  minor  revisions  to  this  document  resulted,  and  are  reflected  in  APPENDIX  C. 

Following  this  review,  each  of  the  PowerPoint  slides  was  displayed  to  the  SMEs.  The  PI 
described  the  symbology  elements  of  each  slide  and  solicited  SME  comments  and 
suggestions  regarding  such  issues  as  readability,  intuitiveness,  clutter,  data  availability, 
adherence  to  established  requirements,  etc.  The  PI  recorded  relevant  comments.  Although 
there  was  a  lively  discussion  on  a  number  of  issues,  and  many  suggestions  documented, 
probably  the  most  significant  finding  was  that  a  combination  of  the  head-referenced 
HMD  and  the  aircraft-referenced  VHUD  presented  some  serious  problems,  particularly  if 
either  of  these  displays  employed  a  distributed  symbology  format.  Symbology 
interactions,  clutter,  and  overwriting  become  problematic  when  the  pilot's  LOS  is  within 
FOV  limits  of  the  VHUD.  Since,  in  general,  the  entire  VHUD  does  not  come  into  view 
instantaneously,  some  symbology/data  elements  of  this  display  are  in  view,  while  others 
are  not.  This  raises  the  issue  of  when  to  "blank"  the  various  head-referenced  data 
elements  provided  on  the  HMD.  The  situation  often  arises  when  either  redundant  data  are 
presented  in  both  reference  systems  (with  the  potential  for  the  over-writing  of  symbology 
or  adding  to  clutter)  or  some  data  element  is  lost  entirely  at  certain  look  angles. 


13 


Since  neither  of  these  conditions  was  considered  acceptable  by  our  SMEs,  an  alternative 
approach  to  be  called  for.  Possibilities  considered  included: 

•  Display  the  VHUD  instantaneously  only  when  all  data  elements  are  within  the  pilot's 
FOV  limits,  while  simultaneously  blanking  redundant  head-referenced  symbology 

•  Eliminate  the  VHUD  altogether 

•  Allow  only  "either/or"  selection  of  head-referenced  symbology  or  the  VHUD 

It  was  decided  that  examples  of  these  approaches  would  be  developed  for  SME  review 
during  the  next  CWA  session. 


5.6.3  CWA  SESSION  3 

The  third  and  final  CWA  session  with  our  pilot  SMEs  was  conducted  on  31  May  2002. 
Last-minute  scheduling  conflicts  and  illness  limited  participation  to  LTC  Draeger  and 
LTC  Simmons.  The  PI  acted  as  moderator  and  also  participated  as  an  additional  SME. 

The  purpose  of  this  session  was  to  review  the  HMD  symbology  concepts  developed  from 
inputs  of  the  earlier  CWA  sessions  to  ensure  compliance  with  SME  recommendations, 
and  also  to  clarify  some  technical  issues  still  outstanding.  One  of  the  primary  issues  to  be 
settled  was  the  mechanization  of  the  VHUD.  To  facilitate  this  discussion  the  problems 
identified  with  the  initial  concept,  principally  the  redundancy  and/or  overwriting  of  data 
elements  between  the  VHUD  and  off-boresight  HMD  symbology,  were  reviewed.  The 
VHUD  concept  was  further  illustrated  by  analyzing  recent  flight  tests  of  a  VHUD 
concept  conducted  by  AFRL,  with  the  help  of  Lockheed  Martin  and  other  contractors. 
Video  flight  test  segments  of  this  work  were  reviewed  via  the  Internet  (AFRL,  2001). 
After  further  discussion,  it  was  decided  that  the  preferred  implementation  of  the  VHUD 
was  to  display  this  symbology  only  when  the  pilot’s  LOS  allowed  for  the  ENTIRE 
VHUD  to  be  viewed  within  the  HMD  FOV.  In  that  case,  all  redundant  display  elements 
of  the  HMD  off-boresight  data  set  would  be  replaced  instantaneously  by  the  VHUD. 
Conversely,  when  the  pilot’s  LOS  would  cause  ANY  data  element  of  the  VHUD  to  be 
attenuated  by  the  boundary  of  the  HMD  FOV,  the  ENTIRE  VHUD  should  be  removed, 
and  replaced  by  the  selected  HMD  off-boresight  symbology  set.  The  optimal  VHUD 
symbology  set  should  subtend  25°-30°,  both  vertically  and  horizontally,  centered  on  the 
aircraft  fuselage  reference  line  (FRL).  It  was  the  opinion  of  the  SMEs  that  a  smaller 
FOV  is  too  restrictive  on  the  pilot’s  head  position,  while  symbology  was  too  spread  out 
for  ease  of  viewing  with  a  larger  FOV. 

HMD  symbology  masking  when  looking  into  the  cockpit  was  also  discussed.  It  was 
agreed  that  the  optimal  approach  would  be  to  permit  pilot  ON/OFF  selection  of  HMD 
symbology  masking,  with  ON  (i.e.,  HMD  DISABLED  when  looking  into  the  cockpit) 
being  the  default.  Control  could  be  either  by  hands-on-throttle-and  stick  (HOT AS)  or 
DVI.  When  selected  OFF  (i.e.,  HMD  symbology  displayed  while  looking  into  the 
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cockpit),  automatic  decluttering  of  the  HMD  symbology  should  be  provided  so  that  only 
limited  graphical  symbology  (i.e.,  horizon,  attitude,  FPM,  targeting  symbology,  etc.) 
remains.  In  general,  no  HMD  alphanumeric  or  digital  symbology  should  be  visible  when 
looking  into  the  cockpit.  This  approach  is  intended  to  minimize  visual  clutter,  while 
affording  the  pilot  the  ability  to  perform  targeting  tasks  that  would  normally  be  inhibited 
because  the  target  could  not  be  seen  through  the  aircraft  structure. 

To  guide  the  discussion  of  the  remaining  technical  points,  a  list  of  issues  had  been 
prepared  and  was  provided  to  the  SMEs.  Various  display  concepts  were  illustrated 
during  the  discussion  primarily  by  PowerPoint  slides  and  reference  to  an  electronic  copy 
of  MIL-STD-1787C.  Some  concepts  were  also  illustrated  by  use  of  a  computer  graphics 
program  provided  by  AFRL,  entitled  “EditPath.”  The  latter  program  enabled  the  PI  to 
demonstrate  a  number  of  display  concepts  and  variations  set  in  motion,  as  though  the 
aircraft  were  performing  aerobatic  maneuvers. 

In  our  original  proposal  for  this  program,  we  had  described  a  “Quick  Look  Evaluation”  to 
provide  insight  into  the  effectiveness  of  various  HMD  symbology  concepts.  The  “rapid¬ 
prototyping  tool”  for  this  portion  of  the  effort  was  to  be  a  state-of-the-art  personal 
computer  system  based  on  the  900MHz  AMD  Duron  processor,  with  dual  17”  monitors 
controlled  by  a  Matrox  G450  dual-head  video  card,  and  game-quality  aircraft  controllers. 
The  proposed  software  for  this  development  effort  was  Shockwave  Director  (Version  8) 
by  Macromedia,  designed  to  facilitate  creation  of  animated  Internet  Web  pages. 

Although  this  portion  of  the  program  was  de-emphasized  at  the  suggestion  of  the 
Government  Program  Manager,  the  methodology  was  evaluated  with  a  view  toward  using 
this  technique  to  provide  realistic  animated  HMD  symbology  to  aid  visualization  and 
discussion  during  CWA  sessions.  Once  the  software  development  had  begun,  however,  it 
was  found  that  Shockwave  Director  represented  anything  BUT  a  “rapid-prototyping 
tool.”  We  assigned  this  project  to  a  veteran  computer  programmer  working  closely  with 
the  PI.  Progress  was  slow  initially,  since  the  programmer  did  not  have  prior  experience 
with  this  software.  It  was  soon  evident,  however,  that  Shockwave  Director,  at  least  the 
latest  version  available  at  the  time  it  was  purchased  (Version  8),  has  serious  limitations 
for  this  application.  We  found  it  virtually  impossible  to  illustrate  3-D  display  concepts, 
such  as  the  Theta  Ball,  in  any  practical  manner,  so  the  effort  was  limited  to  2-D  display 
concepts  only.  Even  this  task,  programmed  in  the  proprietary  Lingo  language,  proved  to 
be  formidable.  For  instance,  it  was  difficult  to  illustrate  even  some  fairly  simple  display 
concepts  at  sufficiently  high  frame  rates  to  avoid  “jerkiness”  on  playback.  The  software 
also  appeared  to  be  quite  unstable;  small  changes  in  one  portion  of  the  code  often  resulted 
in  altered  displays  in  other  unrelated  portions  of  the  program,  and  the  code  performed 
differently  under  different  operating  systems.  In  addition,  compiled  versions  of  the  code 
operated  differently  than  uncompiled  code  on  playback.  Late  in  the  program  we 
theorized  that  some  of  the  instability  and  performance  problems  might  be  the  result  of 
insufficient  memory  hardware,  although  our  development  system  was  equipped  with  a 
respectable  128MB  of  DDR  RAM  (266MHz). 

Several  months  of  programming  were  expended  on  this  effort,  which  finally  resulted  in 
five  “movies”  depicting  an  aircraft  performing  a  canned  series  of  loops  and/or  rolls,  each 
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illustrating  different  HMD  symbology  concepts.  Although  the  quality  of  these 
animations  was  marginal,  they  were  shown  to  the  SMEs  during  the  final  CWA  session 
and  proved  to  be  somewhat  useful  in  aiding  visualization  of  some  display  concepts.  As  a 
general  recommendation  of  this  program,  however,  we  do  NOT  recommend  this 
approach  for  future  development  efforts. 


6.0  HMD  FUNCTIONAL  SPECIFICATIONS 

In  this  section,  each  of  the  data  elements  listed  in  APPENDIX  C  will  be  discussed  in 
detail.  The  order  of  the  discussion  follows  that  of  the  table  in  APPENDIX  C,  and  has  no 
significance.  This  discussion  of  the  features  of  each  HMD  data  element  developed 
during  this  Phase  I SBIR  effort  is  intended  to  provide  a  firm  basis  for  further  evaluation 
during  Phase  II,  or  for  other  follow-on  research. 

Unless  specified  otherwise,  font  size  for  all  alphanumeric  displays  conform  to  MEL-STD- 
1787C  guidance. 


6.1  WEAPON  /  SENSOR  AIMING  REFERENCE 

6.1.1  FUNCTIONALITY 

Probably  the  primary  function  and  advantage  of  the  HMD  in  tactical  aircraft  is  for  the 
aiming  of  weapons  and  sensors,  particularly  at  angles  well  off  the  longitudinal  axis  of  the 
aircraft.  Therefore,  some  sort  of  Aiming  Reference  is  a  priority  for  display  on  the  HMD. 


6.1.2  FORMAT  AND  MECHANIZATION 


A  simple  open  cross  (no  pipper),  similar  to  the  “Gun  Cross”  in  the  F-16  HUD,  was 
preferred  by  our  SMEs.  The  guidance  found  in  MIL-STD-1787C  for  the  size  of  the  cross 
itself  appears  to  be  adequate.  A  suggested  enhancement  was  the  addition  of  a  “FOV 
Box”  surrounding  the  Aiming  Cross  representing  the  FOV  of  the  weapon  or  sensor 
selected.  (In  some  cases  the  “box”  may  actually  be  a  circle  to  indicate  a  circular  FOV.) 
Only  the  comers  of  the  FOV  Box  should  be  depicted  to  reduce  clutter.  An  illustration  of 
this  symbology  is  depicted  in  Figure  la. 


Aiming 

Reference 
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FIGURE  la:  AIMING  REFERENCE 


The  Aiming  Cross  and  the  selected  sensor  FOV  should  move  independently,  so  if  there  is 
any  latency  in  the  sensor’s  reaction  to  pilot  head  movement  the  disparity  between  pilot 
LOS  and  sensor  LOS  should  be  readily  apparent.  As  discussed  in  more  detail  below, 
when  a  sensor  is  locked  on  a  target  it  should  become  earth  stabilized  (fixed  to  the  target 
LOS),  so  that  it  will  be  uncoupled  from  helmet  motion. 

The  Aiming  Cross  should  normally  be  located  about  midway  between  the  center  and  the 
top  of  the  HMD  FOV,  and  decluttered  automatically  when  the  landing  gear  is  down.  It 
would  also  be  desirable,  however,  to  provide  an  option  whereby  the  Aiming  Cross  could 
be  shifted  upward  in  the  HMD  FOV  as  high  as  possible,  controlled  either  by  HOTAS  or 
DVI  (Figure  lb).  There  will  be  situations,  particularly  in  WVR  air-to-air  combat 
engagements,  when  the  pilot  will  not  quite  be  able  to  move  his  head  far  enough  to  place 
the  Aiming  Cross  on  the  target.  The  extra  few  degrees  provided  by  this  option  could  be 
critical  in  this  case. 


Aiming  Reference 


Upper  HMD  FOV 
Boundary 

(not  visible) 

_ 

L,  !  j 


FIGURE  lb:  AIMING  REFERENCE  (shifted  upward) 


Whenever  a  weapon  or  sensor  with  off-boresight  capability  is  selected,  the  Aiming  Cross 
should  be  head  stabilized  on  the  HMD.  To  avoid  conflicts  in  aiming  symbology  in  this 
case,  no  Aiming  Cross  should  be  included  on  the  VHUD,  but  should  remain  visible  on 
the  HMD,  and  occlude  VHUD  data  elements  when  appropriate.  For  instance,  as  the  pilot 
scans  from  side-to-side  across  the  VHUD  FOV,  the  VHUD  symbology  will  remain  fixed 
(aircraft  referenced)  while  the  Aiming  Cross  will  move  across  the  VHUD  FOV  with  the 
pilot’s  LOS.  One  exception  to  this  rule  is  necessary  for  fixed-boresight  weapons  (such  as 
the  gun),  in  which  case  the  Aiming  Cross  should  be  aircraft-referenced  and  displayed 
only  on  the  VHUD  to  indicate  the  boresight  of  the  weapon.  Another  exception  is  made 
for  ballistic  (free-fall)  weapons.  In  this  case,  an  aircraft  FRL  reference  is  useful  for  initial 
estimation  of  the  “aim-of  point”  during  dive-bombing. 
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A  suggested  enhancement  to  the  Aiming  Cross  and  FOV  Box  is  color-coding  to  indicate 
variations  in  weapon  or  sensor  modes. 

6.2  HEADING  AND  COURSE 

6.2.1  FUNCTIONALITY 

Magnetic  Heading  and  Magnetic  Course  are  critical  parameters  for  many  aviation 
navigational  tasks.  As  such,  these  data  elements  should  be  available  to  the  pilot  at  all 
times,  but  they  are  typically  referenced  only  for  navigation  purposes.  Access  to  these 
data  elements  is  seldom  time  critical,  and  rarely  required  when  the  pilot  is  looking  off 
boresight.  Therefore,  these  data  elements  are  prime  candidates  for  display  on  an  aircraft- 
referenced  VHUD.  On  the  VHUD  Magnetic  Heading  and  Course  should  be  selectable  by 
the  pilot,  and  would  be  visible  only  when  viewing  forward  along  the  longitudinal  axis  of 
the  aircraft,  as  described  in  Paragraph  5.6.3  above. 

6.2.2  FORMAT  AND  MECHANIZATION 

The  SMEs  in  this  study  preferred  the  standard  analog  “heading-scale”  format  for 
Magnetic  Heading  and  Course,  normally  located  at  the  top  center  of  the  VHUD  only. 

The  position  of  the  heading  scale  can  shift  to  the  bottom  center  of  the  VHUD 
automatically  with  landing  gear  selected  down,  as  in  current  F-16  HUD  practice.  As 
depicted  in  Figure  2,  precise  current  Magnetic  Heading  or  Course  (as  selected),  to  the 
nearest  whole  degree,  should  be  shown  in  a  box  in  the  center  of  the  scale,  with  small 
scale  marks  at  5°  and  large  marks  at  10°  increments,  labeled  above  the  10°  tics  by  two 
numbers  representing  the  first  two  numbers  of  the  heading,  extending  15°  to  either  side  of 
current  Magnetic  Heading/Course.  Magnetic  Heading/Course  should  be  shown  to 
increase  to  the  right  of  center,  and  decrease  to  the  left.  The  “current  heading  box”  in  the 
center  of  the  display  should  occlude  scale  markings  and  labels.  Standard  5:1 
compression  of  the  heading  scale  is  acceptable.  The  color  of  all  elements  of  this  display 
should  be  the  standard  green.  Selectivity  of  this  display  to  provide  other  references,  such 
as  True  Heading,  or  True  Course,  for  instance,  were  not  considered  critical,  as  long  as 
these  were  provided  in  head-down  displays  (HDDs).  A  “Command  Heading”  indicator, 
such  as  a  caret  below  the  heading,  should  also  be  provided,  as  well  as  “Tadpole” 
Command  Heading  symbology  (as  in  the  F-16),  since  the  latter  provides  an  indication  of 
angular  distance  to  the  commanded  heading  when  this  heading  is  outside  the  limits  of  the 
heading  scale.  This  symbology  is  depicted  in  Figure  2.  In  NAV  mode,  Course  Deviation 
Indicator  (CDI)  symbology  was  not  considered  critical  for  display  on  the  HMD  as  long  as 
it  is  available  on  a  HDD.  If  displayed  on  the  HMD  it  should  be  limited  to  the  VHUD. 

CDI  or  flight-director  symbology,  as  provided  in  MIL-STD-1787C,  should  be  limited  to 
the  VHUD  and  HDDs. 
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FIGURE  2:  HEADING  /  COURSE  DISPLAY 


6.3  VELOCITY  VECTOR,  FLIGHT  PATH  MARKER,  CLIMB/DIVE  MARKER 
6.3.1  FUNCTIONALITY 

It  has  become  common  practice  on  modem  HUDs  to  provide  an  indication  of  the  position 
of  the  aircraft’s  velocity  vector  in  the  form  of  an  FPM.  This  display  element  has  become 
invaluable  to  provide  the  pilot  with  a  prediction  of  future  aircraft  movement.  Placing  the 
FPM  on  the  horizon,  for  instance,  assures  the  pilot  that  the  aircraft  is  in  level  flight, 
regardless  of  speed,  weight,  power  level,  etc.,  and  takes  much  of  the  guesswork  out  of 
flying  high-performance  aircraft.  Recently,  a  modification  of  the  FPM,  called  the  CDM, 
has  been  suggested  as  an  enhancement  or  supplement.  MIL-STD-1787C  describes  in 
great  detail  the  implementation  of  the  CDM,  which  is  essentially  a  FPM  “quickened”  in 
pitch  to  provide  finer  control,  and  constrained  in  yaw  to  remain  on  the  centerline  of  the 
HUD. 


6.3.2  FORMAT  AND  MECHANIZATION 

Since  the  primary  purpose  of  the  CDM  is  to  improve  fine  aircraft  control,  and  the  pilot  is 
likely  to  be  looking  forward  during  times  when  such  fine  control  is  critical,  the  SMEs  in 
this  study  recommend  that  the  CDM  be  provided  as  an  option  for  display,  confined  to  the 
VHUD  only.  Format  and  mechanization  guidance  provided  in  MIL-STD-1787C  was 
considered  sufficient  for  this  purpose. 


* 
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This  study  also  recommends  retaining  the  FPM  for  further  reference.  By  definition,  the 
FPM  is  earth  referenced,  and  should  be  visible  on  the  HMD  whenever  its  position  lies 
within  the  FOV  of  the  HMD.  The  SMEs  in  this  study  did  not  consider  it  necessary  to 
display  a  “ghost”  FPM,  normally  flashing  at  the  limit  of  the  HUD  FOV  in  HUD-equipped 
aircraft,  when  the  FPM  lies  outside  the  FOV  limits  of  the  HMD. 


6.4  ATTITUDE 

6.4.1  FUNCTIONALITY 

Aircraft  Attitude  awareness  is  critical  to  the  pilot  for  aircraft  control,  and  is  an  essential 
element  of  SA.  A  strong  distinction  was  made,  however,  between  attitude  display 
requirements  for  fine  aircraft  control,  as  might  be  necessary  for  instrument  approaches 
and  landings,  and  “attitude  awareness,”  which  requires  much  less  precision.  For  tasks 
requiring  fine  control,  and  therefore  high  attitude-display  precision,  our  SMEs  expect  to 
be  looking  forward  relative  to  the  aircraft.  Consequently,  “high-precision”  attitude 
display  formats  should  be  available  on  the  VHUD,  and  can  be  limited  to  that  display 
technique. 

6.4.2  FORMAT  AND  MECHANIZATION 

To  afford  high  precision  for  tasks  requiring  such  precision,  the  SMEs  preferred  a 
distributed  pitch  ladder  on  the  VHUD.  Mechanization  would  be  as  specified  in  MTT- 
STD-1787C.  In  addition,  the  pitch  ladders  and  scale  numbers  representing  diving 
attitudes  could  be  colored  brown  for  greater  recognition.  The  selected  shade  of  brown  for 
this  enhancement  requires  careful  study,  however,  to  ensure  its  visibility  against  all 
backgrounds.  No  preference  was  shown  for  the  inclusion  of  a  CDM  in  addition  to  the 
standard  FPM.  The  “W”  (waterline)  symbology,  as  described  in  MIL-STD-1787C,  was 
preferred  to  denote  the  aircraft’s  FRL.  There  was  also  some  interest  in  providing 
selectable  attitude  formats  for  the  VHUD,  which  might  include  the  ASAR,  Theta  Ball, 
etc.  (Geiselman  &  Osgood,  1993).  All  attitude  symbology,  whether  on  the  VHUD  or  the 
HMD  in  general,  should  be  subject  to  a  declutter  option. 

When  viewing  at  high  off-boresight  angles,  our  SMEs  chose  to  trade  precision  for 
reduced  clutter;  so  the  Non-Distributed  Flight  Reference  (NDFR)  technique  was 
preferred.  The  NDFR  format  most  preferred  by  our  SMEs  for  display  on  the  HMD  at 
high  off-boresight  viewing  angles  was  the  Arc  Segmented  Attitude  Reference  (ASAR), 
sometimes  referred  to  as  the  “Orange  Peel”  (MIL-STD-1787C).  This  format,  depicted  in 
Figure  3a,  seems  to  offer  an  optimal  compromise  between  precision  and  clutter  for  off- 
boresight  tasks,  and  has  been  shown  in  research  to  be  effective  in  “gross”  aircraft  control 
tasks,  maintaining  attitude  awareness,  and  initial  unusual-attitude  (UA)  recognition  and 
recovery  (Geiselman,  Havig,  &  Brewer,  2000;  Jenkins,  et.  al,  2001;  DeVilbiss  &  Sipes, 
1995;  Drewery,  Davy,  and  Dudfield,  1997).  Supporting  these  functions  with  minimal 
visual  clutter  was  considered  to  be  the  primary  purpose  of  an  off-boresight  HMD  attitude 
reference.  In  addition  to  the  standard  ASAR  mechanization  described  in  M3L-STD- 
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1787C,  the  SMEs  were  provided  with  a  description  and  demonstration  of  an  enhanced 
AS  AR  mechanization  that  includes  a  continuous  wings-level  reference  (Figures  3b-d). 
Our  SMEs  preferred  this  mechanization  over  that  of  the  standard  MEL-STD-1787C 
ASAR.  Further  possible  enhancements  to  the  standard  ASAR  include  a  dashed  arc  for 
dives,  as  opposed  to  a  solid  arc  for  climbs,  and  a  color  change  in  the  arc  from  the 
standard  green  to  red  for  dives  of  greater  than  10°.  These  concepts  are  illustrated  in 
Figure  3d.  There  was  some  concern,  however,  that  these  enhancements  may  prove  to  be 
unnecessary  and  even  distracting,  so  further  evaluation  is  recommended  on  this  issue. 
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FIGURE  3a:  ASAR  ATTITUDE  REFERENCE 
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FIGURE  3b:  ASAR  -  WINGS-LEVEL  CLIMB 
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FIGURE  3c:  ASAR  -  WINGS-LEVEL,  STEEP  CLIMB 
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FIGURE  3d:  ASAR  -  WINGS-LEVEL  DIVE 


A  further  significant  modification  recommended  by  this  study  was  to  mechanize  the 
ASAR  roll-attitude  display  to  provide  an  “outside-in”  perspective.  Outside-in  attitude 
indicators  of  various  designs  have  been  studied  since  the  1930s,  and  have  been  standard 
for  Russian-built  aircraft  for  many  years  (Previc  &  Ercoline,  1998).  These  displays  have 
consistently  out-performed  inside-in  displays,  standard  in  the  West,  for  intuitiveness  and 
unusual-attitude  recoveries.  One  of  the  major  drawbacks  of  the  outside-in  perspective  is 
that  such  displays  cannot  me  made  conformal  with  the  outside  world,  so  they  would 
likely  be  confusing  for  use  on  a  HUD  or  HMD  in  visual  conditions  when  looking  forward 
along  the  aircraft’s  axis  with  the  natural  horizon  in  view.  As  recommended  here, 
however,  the  ASAR  would  only  be  displayed  when  the  pilot  is  viewing  well  off 
boresight,  a  condition  in  which  conformal  displays  have  actually  demonstrated  inferior 
performance  to  non-conformal  displays. 
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The  format  recommended  by  this  study  provides  an  outside-in  perspective  in  roll  only. 
The  ASAR  arc  would  be  fixed  along  the  vertical  axis  so  that  the  “gap”  would  appear  only 
at  the  top  or  bottom  of  the  arc.  Roll  is  represented  by  rotating  the  aircraft  symbol  inside 
the  arc,  while  pitch  continues  to  be  depicted  by  the  expansion  and  contraction  of  the 
surrounding  arc,  as  shown  in  Figure  3e.  The  digital  airspeed  and  altitude  windows  in  this 
format  are  placed  just  outside  the  ASAR  arc  on  their  respective  sides  to  avoid  conflict 
with  the  rotating  aircraft  symbol. 
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FIGURE  3e:  ASAR  -  DIVING  TURN 
(Outside-In  Perspective  in  Roll  Axis  Only) 


Placement  of  the  ASAR  should  be  low  center  of  the  HMD  (head-referenced)  display. 

The  diameter  of  the  ASAR  circle  should  be  kept  as  small  as  practical  (40-50  mr 
suggested).  Placement  of  the  ASAR  is  based  on  research  that  indicates  improved 
response  to  graphic  displays  and  motion  in  the  lower  visual  field,  using  peripheral  vision, 
while  focused  higher  in  the  visual  field,  rather  than  vice  versa  (Sochacki  &  Wickens, 
1997;  Previc,  2000).  This  placement  should  optimize  the  pilot’s  ability  to  notice 
unintended  attitude  changes  in  the  ASAR  while  concentrating  on  other  areas  of  the  HMD 
FOV  that  require  focused  attention,  such  as  placement  of  the  Aiming  Reference  over  a 
target  or  reading  alphanumeric  displays. 

We  recommend  that  the  ASAR  pitch  attitude  reference  be  based  on  actual  aircraft  attitude 
(Fuselage  Reference  Line),  rather  than  flight  path  as  suggested  by  Geiselman,  1999, 
without  explanation.  This  recommendation  reflects  the  primary  purpose  of  this  display  at 
high  off-boresight  viewing  angles,  namely  attitude  awareness  rather  than  fine  aircraft 
control.  There  is  some  concern,  particularly  for  new  and  future  aircraft  with  extremely 
large  angle-of-attack  range,  that  a  flight  path-based  reference  could  result  in  a  mental 
disconnect  between  the  display  and  the  pilot’s  expectations  and  perceptions.  Further 
study  is  recommended  on  this  issue. 

In  addition  to  the  ASAR  NDFR  attitude  display,  the  SMEs  strongly  endorsed  the 
inclusion  of  a  full-FOV  conformal  horizon  for  the  HMD.  Extended  conformal  horizon 
symbology  was  thought  to  be  very  valuable  in  maintaining  attitude  awareness  while 
viewing  at  high  off-boresight  angles.  This  symbology  should  be  earth  referenced,  and 
overlie  the  actual  horizon  whenever  the  HMD  FOV  includes  the  horizon.  A  “horizon 
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ine  should  extend  to  the  limits  of  the  HMD  FOV  in  any  direction,  depending  on 
orientation.  A  minimum  of  five  (5)  short  “perspective  lines”  should  be  attached  to  the 
lower  side  of  the  horizon  line  to  differentiate  UP  from  DOWN.  Whenever  the  center  of 


the  horizon  line  lies  outside  the  HMD  FOV,  the  conformal  horizon  symbology  should 
peg  at  this  limit,  so  that  at  least  a  quarter  of  the  symbology,  depending  on  orientation, 
remains  visible  and  parallel  to  the  actual  horizon  at  all  times.  When  pegged,  the  entire 
symbology  should  be  dashed,  otherwise  it  should  be  composed  of  solid  lines.  The  color 
of  the  conformal  horizon  line  should  be  brown.  Selection  of  the  shade  of  brown  is 
critical  to  ensure  visibility  under  all  background  conditions.  These  concepts  are 
illustrated  in  Figures  4a  and  4b.  Portions  of  the  conformal  horizon  symbology  that 
overlie  the  VHUD  FOV  should  be  occluded  by  the  VHUD,  which  may  provide  its  own 
attitude  reference  (i.e.,  pitch  ladder). 


FIGURE  4a:  ASAR  WITH  CONFORMAL  HORIZON 
[Looking  left  along  wing  line  (slightly  high)  with  aircraft  straight  and  level] 

FIGURE  4b:  ASAR  WITH  CONFORMAL  HORIZON 
[Looking  left  along  wing  line  with  aircraft  in  30°  dive  and  40°  left  bank] 
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It  is  recognized  that  implementation  of  the  conformal  horizon  concept  as  described  may 
be  problematic  given  our  current  understanding  of  the  limitations  of  Strike  Helmet  21. 
However,  these  specifications  are  provided  as  a  goal. 

6.5  AIRSPEED 

6.5.1  FUNCTIONALITY 

Airspeed  awareness  is  considered  to  be  critical  to  the  pilot  for  aircraft  control,  and  is  an 
essential  element  of  SA.  Therefore,  it  should  be  available  to  the  pilot  at  all  times. 

6.5.2  FORMAT  AND  MECHANIZATION 

In  keeping  with  the  SMEs’  preference  of  the  NDFR  for  high  off-boresight  viewing 
angles,  a  digital  display  of  airspeed  (to  within  1  kt),  was  considered  optimal  for  a  head- 
referenced  display,  visible  at  all  times  to  the  pilot.  This  data  element  should  be  located 
just  outside  the  ASAR  circle,  to  the  left  and  slightly  below  center.  The  color  of  this 
display  should  ordinarily  be  standard  green,  but  may  change  to  red  and  flash  at  about  a  4- 
Hz  rate  to  indicate  critical  airspeed  levels,  or  limits.  These  limits  should  be 
programmable  by  the  pilot  during  preflight.  In  addition  to  the  visual  cue  of  airspeed 
limits,  an  audio/voice  alert  should  also  be  provided.  A  similar  mechanization  is 
suggested  for  Mach  number.  This  digital  display  (accurate  to  0.01  Mach),  should  be 
located  just  below  the  left  of  the  digital  airspeed  display.  This  symbology  is  depicted  in 
Figure  5.  As  with  the  rest  of  the  NDFR  elements,  airspeed  and  Mach  should  be  subject  to 
declutter. 

The  preferred  format  for  airspeed  indication  on  the  VHUD  was  a  distributed  format, 
including  a  round,  analog  dial-type  format  located  to  the  left  of  center  in  the  upper  area  of 
the  VHUD  (Ercoline  &  Gillingham,  1990).  The  dial  would  comprise  a  circle  of  ten  dots 
with  the  current  digital  airspeed  in  the  center  of  the  circle,  and  an  internal  pointer 
“needle.”  The  units  for  airspeed  shown  by  the  dial  display  would  always  be  knots 
calibrated  airspeed  (KCAS).  Digital  Mach  should  be  displayed  just  below  the  airspeed 
dial.  Provision  should  be  made  for  selecting  either  True  Airspeed  or  Ground  Speed  for 
this  display  instead  of  Mach. 
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FIGURE  5:  AIRSPEED  DISPLAY  INCLUDING  MACH  NUMBER 


6.6  ALTITUDE 

6.6.1  FUNCTIONALITY 

Altitude  awareness  is  considered  to  be  critical  to  the  pilot  for  aircraft  control,  and  is  an 
essential  element  of  SA.  Therefore,  it  should  be  available  to  the  pilot  at  all  times.  Two 
measures  of  altitude  are  considered  critical:  barometric  altitude  above  mean  sea  level 
(MSL)  and  altitude  above  ground  level  (AGL).  AGL  altitude  is  commonly  referred  to  as 
“Radar  Altitude,”  since  the  radar  altimeter  is  the  common  means  of  measuring  this 
distance,  but  it  is  recognized  that  this  may  not  always  be  the  case  in  the  future. 

6.6.2  FORMAT  AND  MECHANIZATION 

In  keeping  with  the  SMEs’  preference  of  the  NDFR  for  high  off-boresight  viewing 
angles,  a  digital  display  of  altitude  (to  within  100  ft),  was  considered  optimal  for  a  head- 
referenced  display,  visible  at  all  times  to  the  pilot.  This  data  element  should  be  located 
just  outside  the  ASAR  circle  to  the  right,  slightly  below  center.  The  font  size  for 
Thousands  should  be  discemibly  larger  than  for  Hundreds,  and  a  hyphen  should  separate 
Hundreds  from  Thousands.  The  color  of  this  display  should  be  standard  green.  As  with 
the  rest  of  the  NDFR  elements,  altitude  should  be  subject  to  declutter. 

A  suggested  enhancement  to  this  off-boresight  altitude  symbology  was  to  allow  an  option 
to  have  barometric  altitude  revert  to  absolute  (radar)  altitude  at  a  selected  AGL  altitude. 
The  same  symbology  could  be  used,  but  the  distinction  in  altitude  reference  should  be 
noted  by  the  addition  of  an  “R”  (radar)  or  “A”  (absolute  or  AGL). 

The  preferred  format  for  altitude  indication  on  the  VHUD  was  a  distributed  format, 
including  a  round,  analog  dial-type  format  located  to  the  right  of  center  in  the  upper  area 
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of  the  VHUD  (Ercoline  &  Gillingham,  1990).  The  dial  would  comprise  a  circle  of  ten 
dots  with  the  current  digital  altitude  (to  the  nearest  10  ft)  in  the  center  of  the  circle,  and 
an  internal  pointer  “needle.”  The  font  size  for  Thousands  should  be  discemibly  larger 
than  for  Hundreds  and  Tens  for  altitudes  above  1,000  ft;  below  that  altitude  all  numbers 
would  revert  to  the  larger  font.  The  units  for  altitude  shown  by  the  dial  display  would 
always  be  feet  MSL. 

The  thermometer  scale  was  the  preferred  format  for  display  of  AGL  altitude  on  the 
VHUD.  This  display  should  be  located  in  the  lower  right  portion  of  the  VHUD,  and 
would  disappear  above  1,500  ft  AGL.  A  logarithmic  scale  is  suggested,  as  shown  in 
MIL-STD-1787C,  to  improve  precision  at  low  AGL.  The  precise  radar  altitude  may  be 
provided  digitally  in  a  box  to  the  left  of  the  scale,  vertically  aligned  with  the  top  of  the 
red  thermometer  bar,  as  shown  in  Figure  6a.  Further  evaluation  was  suggested  to 
determine  whether  the  additional  digital  value  is  required.  The  AGL  altitude  scale  should 
be  subject  to  declutter  options,  including  the  option  to  decluttter  automatically  when  the 
landing  gear  is  down.  The  capability  to  replace  this  display  with  a  simple  digital  readout, 
located  just  below  the  altitude  dial,  was  also  recommended,  at  least  then  above  1,500  ft 
AGL,  below  which  the  thermometer  scale  appears.  This  readout  should  be  labeled  with 
an  “R”  (for  Radar)  or  “A”  (for  Absolute  or  AGL),  and  10-ft  resolution  should  be 
adequate. 

As  the  primary  function  of  the  a  radar  altitude  display  on  the  HMD  at  large  viewing 
angles  off  boresight  is  to  alert  the  pilot  of  potentially  dangerous  low-altitude  situations 
and  to  provide  trend  information,  rather  than  for  precise  positioning  of  the  aircraft.  A 
thermometer  display  format  was  also  recommended  for  this  purpose  in  order  to  exploit 
the  natural  superiority  of  this  type  display  for  trend  information.  As  with  the  VHUD 
display  of  AGL  altitude,  the  head-referenced  symbology  should  be  located  in  the  lower 
portion  of  the  HMD  display,  to  the  right  of  center,  and  should  be  subject  to  a  declutter 
option.  Optimal  offset  distance  to  the  right  requires  further  evaluation,  but  about  30°  is 
suggested.  When  selected,  the  AGL  altitude  display  would  be  activated  below  1,500  ft 
AGL.  It  is  suggested  that  the  moving  thermometer  element  of  this  display  be  colored  red 
for  its  alerting  value  on  activation.  The  scale  itself,  as  depicted  in  Figure  6b,  should  be 
linear  to  provide  a  more  intuitive  assessment  of  closure  with  the  ground,  and  colored 
green.  The  height  of  the  scale  should  be  no  greater  than  half  the  vertical  FOV  of  the 
HMD.  Labels  and  scale  tic  marks  should  be  provided  only  each  500  ft  to  reduce  clutter 
and  improve  readability  while  allowing  minimal  display  size. 
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FIGURE  6a:  LOGARITHMIC  RADAR  ALTIMETER  SCALE 
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FIGURE  6b:  HMD  WITH  RADAR  ALTIMETER  DISPLAY 
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6.7  VERTICAL  VELOCITY 


6.7.1  FUNCTIONALITY 

Vertical  Velocity  is  required  by  MEL-STD-1787C  for  Primary  Flight  Reference  (PFR),  so 
it  should  be  available  to  the  pilot.  Its  primary  utility  is  to  facilitate  fine  control  of  descent 
rate,  especially  during  approach  and  landing,  and  as  an  aid  in  anticipating  altitude  trends 
during  instrument  flight.  Normally  these  functions  are  served  more  effectively  in  modem 
aircraft  by  the  FPM  or  CDM.  Therefore,  this  data  element  was  considered  to  be  of  low 
priority  by  our  SMEs. 

6.7.2  FORMAT  AND  MECHANIZATION 

The  SMEs  in  this  study  preferred  the  analog  arc -type  format  for  Vertical  Velocity, 
attached  to  the  left  hemisphere  of  the  altitude  dial  display  on  the  VHUD,  as  illustrated  in 
Figure  7.  The  length  of  the  arc  should  increase  from  the  9  o’clock  position  clockwise 
around  the  altimeter  scale  with  increasing  altitude,  and  vice  versa.  Zero  climb  rate  would 
be  located  at  9  o’clock,  with  each  “100-ft  dot”  on  the  altimeter  representing  1,000  ft/min 
of  climb  or  descent  rate.  So  the  800-ft  dot  would  indicate  500  ft/min  climb  rate,  the  900- 
ft  dot  indicates  1,500  ft/min  climb,  and  the  12  o’clock  position  2,500  ft/min  climb  rate. 
The  Vertical  Speed  Arc  can  “peg”  at  the  9  o’clock  position  in  either  direction  when 
climb/dive  rate  exceeds  5,000  ft/min.  The  color  of  the  Vertical  Velocity  arc  should  be 
standard  green.  No  display  of  Vertical  Velocity  is  required  on  the  HMD  when  viewing  at 
large  off-boresight  angles;  the  Vertical  Velocity  display  can  be  limited  to  the  VHUD 
alone,  and  should  be  subject  to  declutter. 


Vertical  Speed  Arc 

(1,500  ft/min  Climb)  "  * 

'  i  .. 

"  £>■?.  .  . 
t.  V'“: 

tu  t'  » 


FIGURE  7:  VERTICAL  SPEED  DISPLAY 


6.8  ACCELERATION 
6.8.1  FUNCTIONALITY 

When  precision  in  airspeed  control  is  required,  it  is  often  helpful  to  have  a  direct 
indication  of  aircraft  acceleration/deceleration.  Small  accelerations  are  often  difficult  for 
the  pilot  to  sense  by  “seat-of-the-pants,”  so  attention  is  required  to  monitor  airspeed  over 
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extended  periods  of  time  to  detect  speed  changes.  This  monitoring  function  increases 
workload,  and  speed  changes  can  only  be  detected  “after-the-fact.”  Changes  in  speed  can 
be  anticipated,  and  speed  control  can  be  smoother  and  more  precise,  if  the  pilot  is 
provided  with  an  early  indication  of  instantaneous  trends  in  airspeed.  Precise  speed 
control  is  important  in  the  approach  and  landing  phase  of  flight.  It  can  also  be  of  value  in 
setting  cruise  power  for  optimal  conditions. 

6.8.2  FORMAT  AND  MECHANIZATION 

The  Acceleration  Cue  format  and  mechanization  provided  in  MIL-STD-1787C,  was 
considered  to  be  adequate  for  this  display.  It  provides  for  a  caret  referenced  to  the  left 
“wing”  of  the  CDM  (the  FPM  could  also  be  used);  caret  position  above  the  reference 
indicates  positive  longitudinal  acceleration,  while  caret  position  below  the  reference 
represents  deceleration.  The  Acceleration  Cue  should  be  limited  to  the  VHUD  only. 


6.9  LOAD  FACTOR 

6.9.1  FUNCTIONALITY 

Load  factor  (G)  is  a  critical  flight  parameter  in  fighter  aviation  with  implications  for  both 
aircraft  and  pilot  performance.  Its  primary  function  is  to  provide  a  means  of  adhering  to 
the  design  structural  limitations  of  the  aircraft  under  varying  load  configurations.  With 
the  advent  of  modem  fly-by- wire  flight-control  systems,  this  function  has  sometimes 
been  accomplished  more  recently  by  programmed  flight-control  limiters,  but  in  most 
cases  compliance  with  some  load-factor  limitations  continues  to  be  left  to  the  pilot.  In 
the  situation  where  all  conceivable  load-factor  limitations  are  satisfied  by  automatic 
flight-control  limiters,  it  is  difficult  to  see  the  need  for  a  load-factor  display.  Still,  the 
SMEs  participating  in  this  study  were  reluctant  to  eliminate  this  display  parameter, 
probably  because  of  years  of  dependence  on  it.  Possibly  at  some  future  time  this 
parameter  may  be  considered  dispensable.  Currently,  however,  this  study  recommends 
that  it  continue  to  be  made  available  to  the  pilots  of  tactical  aircraft. 

6.9.2  FORMAT  AND  MECHANIZATION 

Load  factor  has  traditionally  been  provided  by  analog  dial-type  instruments  in  HDDs,  and 
more  recently  by  simple  digital  readouts  on  HUDs.  The  SMEs  in  this  study  preferred  the 
digital  format  for  application  to  the  HMD.  As  described  in  MIL-STD-1787C,  Current  G 
would  be  displayed  in  the  top  position  of  the  data  stack  described  in  Section  6.13  below, 
which  also  contains  Weapon  Selection,  and  Aircraft  Master  Mode.  This  data  stack 
should  be  located  in  the  lower  portion  of  the  HMD  to  the  left  of  center,  and  in  a  similar 
position  on  the  VHUD.  This  display  should  be  subject  to  a  declutter  option.  A 
recommended  enhancement  is  to  provide  a  high-G  audio  cue,  similar  to  the  one  described 
for  AOA  above,  to  alert  the  pilot  of  approaching  G  limitations. 
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6.10  ANGLE-OF-ATTACK  (AOA) 


6.10.1  FUNCTIONALITY 

AOA  is  a  critical  parameter  for  aircraft  performance  and  control  during  many  flight  tasks, 
and  is  included  in  the  PFR  list  of  MIL-STD-1787C.  Therefore,  it  should  be  available  to 
the  pilot  when  needed.  This  display  should  indicate,  as  a  minimum,  variation  in  AOA 
from  the  desired  value. 

6.10.2  FORMAT  AND  MECHANIZATION 

AOA  is  a  valuable  parameter  for  optimizing  many  normal  flight  phases,  including 
approach  and  landing,  and  maximizing  range  and  endurance.  For  such  purposes,  display 
only  on  the  VHUD  is  adequate.  As  discussed  in  MIL-STD-1787C,  there  are  a  number  of 
AOA  display  approaches  currently  in  use,  including  digital  readouts,  AOA  “worms,” 
“brackets,”  dials,  and  tapes.  The  SMEs  in  this  study  preferred  different  formats  for 
various  flight  tasks.  For  approach  and  landing  (i.e.,  with  landing  gear  down)  the  AOA 
Worm  format  was  preferred,  as  it  seems  to  present  minimum  clutter.  As  illustrated  in 
Figure  8,  this  symbology  comprises  a  vertical  rectangle  extending  from  the  left  “wing”  of 
the  CDM  or  FPM,  with  its  length  proportional  to  AOA  error  from  “on  speed.”  Extension 
above  the  wing  represents  AOA  below  the  target  value  (fast)  and  below  indicates  AOA 
above  the  target  value  (slow). 

Height  of  worm  indicates 
deviation  from  on-speed  AOA 


FIGURE  8:  AOA  WORM  (from  MIL-STD-1787C) 


A  further  strong  recommendation  was  for  inclusion  of  an  AOA  audio  indicator.  This 
indication  could  consist  of  a  series  of  low-pitched  “beeping”  tones  beginning  a  few 
degrees  below  the  target  AOA  value,  with  increasing  frequency  of  the  beeps  as  AOA 
increases,  until  they  become  nearly  a  steady  tone  at  about  half  a  degree  below  the  target 
AOA.  Target  AOA  is  “on  speed”  AOA  with  gear  down.  The  audio  tone  should  be  silent 
within  half  a  degree  of  target  AOA,  but  if  AOA  increases  above  this  level  the  alert  tones 
should  sound  again,  this  time  at  a  higher  pitch,  increasing  in  frequency  until  they  become 
a  constant  tone  about  one  degree  below  stall  AOA,  or  at  limit  AOA  for  AOA-limited 
aircraft. 
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For  clean  maneuvering  there  are  two  critical  AO  As:  maximum-lift  (or  limit)  AOA  and 
AOA  for  maximum  energy-maneuvering  performance  (e.g.,  maximum  sustained  turn 
rate).  It  is  suggested  that  the  “beeping-tone”  approach  be  adopted  for  this  purpose  also, 
with  a  lower-pitch  tone  implemented  for  maximum  sustained-turn  AOA  and  a  higher 
pitch  to  indicate  maximum  lift  (or  limit  AOA)  as  in  the  gear-down  case.  Our  SMEs 
considered  such  an  audio  display  to  be  superior  to  a  visual  display  for  off-boresight 
viewing  angles  and  heavy  maneuvering,  and  no  visual  display  of  AOA  is  considered 
necessary  off-boresight. 


6.11  MASTER  WARNING  /  CAUTION,  ALERTS 

6.11.1  FUNCTIONALITY 

Typically,  the  Warning  /  Caution  Light  panel  in  fighter  aircraft  is  located  well  into  the 
visual  periphery  when  the  pilot  is  looking  forward,  and  may  be  completely  out  of  sight  in 
high  off-boresight  viewing  situations.  Some  means  of  alerting  the  pilot  to  illumination  of 
any  of  these  lights,  which  could  be  an  indication  of  an  emergency  or  abnormal  situation, 
should  be  provided  regardless  of  the  pilot’s  LOS. 

6.11.2  FORMAT  AND  MECHANIZATION 

Our  SMEs  drew  a  sharp  distinction  between  WARNING  indications  and  all  others  (i.e., 
CAUTION,  various  alerts).  They  were  in  agreement  that  WARNING  lights,  generally 
associated  with  more  serious  problems,  should  be  supplemented  by  both  visual  and  audio 
(preferably  artificial  voice)  alerts,  regardless  of  the  pilot’s  LOS.  The  preference  in  this 
study  was  for  a  conspicuous  “WARNING”  flashing  in  red  near  the  bottom  center  of  the 
VHUD  and  the  HMD  (just  above  the  ASAR  symbology)  when  viewing  off-boresight, 
accompanied  by  a  distinctive  audio/voice  alert,  to  draw  the  pilot’s  attention  to  the  head- 
down  WARNING  /  CAUTION  light  panel  for  further  diagnosis.  The  LOW  FUEL  or 
BINGO  alert  was  preferred  only  on  the  VHUD,  accompanied  by  a  distinctive  audio/voice 
alert.  CAUTION  lights  should  be  accompanied  only  by  an  audio/voice  alert.  It  was  also 
suggested  that  voice  CAUTION  alerts  also  be  provided  when  exceeding  carriage  limits 
(airspeed,  G,  etc.)  of  an  onboard  weapon,  and  launch  limits  if  armed.  This  alert  should 
include  the  parameter  being  exceeded,  for  example,  “AIRSPEED  -  WEAPON.”  A 
launch  parameter  limit  should  also  be  indicated  visually,  as  discussed  below  for  air-to-air 
weapons. 

Another  interesting  suggestion  was  to  provide  a  “Virtual  WARNING  /  CAUTION  Light 
Panel.”  This  virtual  display  would  be  aircraft  stabilized,  and  available  on  the  HMD 
whenever  the  pilot  looked  in  a  given  direction  in  the  cockpit,  for  instance  down  and  right 
where  head-down  WARNING  /  CAUTION  Light  Panels  are  often  located. 
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6.12  HEAD  ORIENTATION 


6.12.1  FUNCTIONALITY 

Our  SMEs  considered  it  desirable  to  have  an  indication  of  the  pilot’s  LOS  relative  to  the 
outside  world  at  all  times.  The  possible  value  of  this  parameter  can  be  appreciated,  since 
it  could  contribute  significantly  to  the  pilot’s  ability  to  describe  the  position  of  an  object 
detected  visually  to  a  wingman  (i.e.,  “Target  bearing  350,  up  10°”).  In  addition,  target 
bearings  from  outside  sources  are  often  relayed  to  the  pilot  in  terms  of  magnetic  bearing, 
so  increased  precision  in  determining  this  parameter  may  aid  in  visual  detection. 

6.12.2  FORMAT  AND  MECHANIZATION 

To  be  of  value,  this  information  element  should  be  head-referenced  and  displayed  on  the 
HMD  at  all  LOS  angles,  except  when  occluded  by  the  VHUD.  The  preferred  format  was 
an  alphanumeric  display  in  the  upper  left-hand  comer  of  the  HMD  FOV  in  two  lines  of 
characters,  one  above  the  other.  As  depicted  in  Figure  9,  the  top  display  would  include 
three  numbers  representing  magnetic  bearing  of  the  helmet  LOS  (i.e.,  the  Aiming 
Reference)  to  the  nearest  degree.  The  lower  display  would  include  1-3  numbers 
representing  helmet  LOS  elevation  angle  to  the  nearest  degree,  followed  by  the  characters 
“U”  or  “D,”  for  Up  or  Down,  respectively.  This  display  should  be  standard  green  in 
color  and  subject  to  declutter. 

The  possibility  of  indicating  HMD  azimuth  LOS  using  a  heading  scale  (as  described  in 
Section  6.2  above  for  the  VHUD)  tied  to  HMD  movement  was  also  raised.  This 
approach  would  require  further  study  to  determine  its  effectiveness  and  desirability. 


HMD  FOV 


33 


6.13  AIRCRAFT  MASTER  MODE,  WEAPON  SELECTION,  SAFE  /ARM 
6.13.1  FUNCTIONALITY 


Modem  digital  aircraft  systems  are  routinely  configured  electronically  to  facilitate 
optimal  performance  for  multiple  roles  and  missions.  Typical  modes  include  NAV  for 
general  cruise,  Air-to-Air  (A/A)  for  air-to-air  combat,  and  a  variety  of  Air-to-Ground 
(A/G)  weapon-delivery  modes,  including  Continuously  Computed  Impact  Point  (CCIP), 
Continuously  Computed  Release  Point  (CCRP),  etc..  In  most  cases  a  modem  combat 
aircraft  also  carries  multiple  weapon  types,  so  some  indication  is  necessary  of  this 
selection  status  and  the  number  of  weapons  remaining.  In  addition,  pilots  of  combat 
aircraft  have  the  ability  to  “safe”  or  “arm”  onboard  weapons  as  the  situation  requires. 

The  SMEs  in  this  study  considered  these  functions  to  be  of  sufficient  importance  to  merit 
off-boresight  indication  to  the  pilot. 

6.13.2  FORMAT  AND  MECHANIZATION 

A  digital  alphanumeric  format  was  preferred  by  our  SMEs  for  these  functions.  At  the  top 
of  the  “data  stack”  would  be  Current  G,  as  discussed  in  Section  6.9  above.  Load  Factor 
should  be  provided  to  the  nearest  0.1G,  followed  by  “G.” 

Listed  second  in  this  data  stack  would  be  Master  Mode.  A  single  letter  (i.e.,  “N,”  “A,”  or 
“G”)  should  suffice  to  reduce  clutter.  Sub-modes  could  be  indicated  by  replacing  the 
Master  Mode  with  the  Sub-mode  abbreviation  (i.e.,  CCIP,  CCRP,  etc.) 

Last  in  the  stack  would  be  weapon  selection/availability,  indicated  by  the  number  of 
weapons  of  the  selected  type  available,  followed  by  the  abbreviation  of  the  weapon. 
Examples  might  be  “4  SW”  for  four  SIDEWINDER  A/A  missiles  or  6  MAV  for  six 
MAVERICK  A/G  missiles  available  and  selected. 

The  lower  portion  of  the  HMD,  to  left  of  center,  was  suggested  as  the  optimal  location  for 
these  data,  as  illustrated  in  Figure  10.  This  entire  data  stack  should  be  subject  to 
declutter.  Similar  data,  format,  and  mechanization  were  recommended  for  the  VHUD. 

For  the  weapons  SAFE  and  ARM  condition,  it  was  thought  better  to  place  the  words 
“SAFE”  or  “ARM,”  as  appropriate,  just  beneath  the  Aiming  Reference,  in  a  more  visible 
position.  For  added  emphasis  SAFE  could  be  colored  green,  and  ARM  red.  These 
indications  would  not  be  required  when  no  weapon  is  selected. 
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FIGURE  10:  HMD  WITH  WEAPON  DATA  STACK 


6.14  NAVIGATION  DATA 

6.14.1  FUNCTIONALITY 

Modem  aircraft  navigation  systems  typically  provide  valuable  data  to  the  pilot  to 
facilitate  location  awareness  and  increase  navigation  accuracy  and  timing  precision, 
which  may  be  critical  to  mission  success.  Typical  data  provided  include  navigation 
waypoint  selected,  distance  to  waypoint,  time  over  waypoint,  time-to-go  to  waypoint,  etc. 
In  addition,  the  navigation  system  may  display  a  preplanned  course,  or  indicate 
deviations  from  that  course. 

6.14.2  FORMAT  AND  MECHANIZATION 

Our  SMEs  preferred  a  digital  alphanumeric  format  for  much  of  these  data.  In  addition, 
since  most  navigation  functions  can  be  suspended  temporarily  for  off-boresight  viewing, 
it  was  recommended  that  alphanumeric  data  be  limited  to  the  VHUD.  The  preferred 
location  was  the  lower  portion  of  the  VHUD,  to  the  right  of  center.  MIL-STD-1787C 
guidance  was  considered  sufficient  for  this  display.  All  these  data  should  be  subject  to  a 
declutter  option. 

In  addition  to  such  traditional  (“first-order”)  navigational  data,  our  SMEs  consider  the 
availability  of  “second-order”  navigational  timing  indicators  to  be  extremely  valuable. 
Here  we  refer  to  first-order  data  as  an  indication  of  “where  we  are,”  i.e.,  number  of  NM 
or  minutes  from  target.  Second-order  navigational  data  would  indicate  “how  we  get 
there;”  in  particular,  how  to  get  to  a  given  waypoint  at  a  specific  time.  Probably  the 
optimal  method  of  accomplishing  this  function  is  to  provide  a  calculated  speed  to  be 
flown  in  order  to  arrive  over  the  selected  waypoint  at  the  time  programmed  into  the 
navigation  system.  A  format  and  mechanization  such  as  the  Commanded  Airspeed  Caret 
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described  in  MIL-STD-1787C,  located  adjacent  to  the  outside  of  the  VHUD  airspeed 
dial,  was  thought  to  be  optimal.  Properly  labeled,  this  caret  could  also  provide 
performance  measures  such  as  speed  for  maximum  range,  endurance,  datalink 
commanded  airspeed,  etc.  Again,  it  is  recommended  that  this  symbology  be  limited  to 
the  VHUD. 


6.15  FLIGHT  PATH  GUIDANCE 

6.15.1  FUNCTIONALITY 

Certain  flight  tasks  have  been  shown  to  benefit  from  navigational  guidance,  which  may 
take  the  form  of  an  indication  of  deviation  from  a  planned  course  or  altitude,  or  a 
“second-order”  display  indicating  the  optimal  maneuver  for  returning  to  the  desired 
course  or  altitude,  as  in  a  flight  director. 

6.15.2  FORMAT  AND  MECHANIZATION 

For  basic  course  deviation,  our  SMEs  considered  the  typical  CDI  implemented  on  a  HDD 
to  be  adequate  and  optimal.  For  instrument  approach  purposes,  the  options  of  analog 
CDI  and/or  flight-director  symbology  on  the  VHUD,  and  a  “pathway-in-the-sky”  is 
suggested.  The  dual-cue  flight-director  symbology  and  mechanization  described  in  MTT  - 
STD-1787C  for  the  F-22  fighter  was  preferred  for  that  display  technique.  Research  has 
been  conducted  on  pathway-in-the-sky  symbology  since  the  early  1950s,  first  for  HDDs, 
and  more  recently  with  HUDs  (Fadden,  Ververs,  and  Wickens,  2000;  Barrows,  Alter, 
Enge,  Parkinson,  Powell,  1998;  Reising,  et.  ah,  1995;  Ververs  &  Wickens,  1998).  There 
can  be  little  reasonable  doubt  that  this  symbology  has  significant  benefits  for  improving 
precision  and  SA,  with  possible  costs  of  increased  visual  clutter  and  “attentional 
tunneling,”  at  least  in  the  instrument-approach  environment.  Still,  while  flight  directors 
have  proven  to  be  effective  for  many  years,  the  operational  benefits  of  pathway-in-the- 
sky  symbology  has  yet  to  be  demonstrated  in  day-to-day  flight  operations.  It  seems 
reasonable  that  personal  preference  may  be  the  deciding  factor  in  the  choice  of 
techniques  for  this  function,  but  it  appears  that  the  time  has  come  to  provide  at  least  the 
option  of  pathway-in-the-sky  symbology. 

A  number  of  proposed  pathway  display  formats  were  considered,  including  a  “highway- 
in-the-sky”  composed  of  pathway  “blocks”  drawn  in  perspective  along  with  a  “follow-me 
aircraft”  symbol  to  provide  an  altitude  cue  and  speed  guidance  (Reising,  et.  al.,  1995), 
and  both  closed  (Barrows  et.  al,  1998)  and  open  “tunnel-in-the-sky”  (Fadden  et.  al,  2000). 
The  latter  technique  (i.e.,  tunnel-in-the-sky  with  open  top)  was  preferred  in  this  study, 
primarily  due  to  the  minimization  of  clutter.  As  recommended  in  the  research  (Reising, 
Liggett,  and  Hartsock,  1995),  extension  of  the  tunnel  forward  to  a  point  representing 
about  45  secs  along  the  aircraft’s  flight  path  seemed  reasonable.  Distance  between  the 
individual  “gates”  should  represent  about  10  secs  of  flight  for  straight  segments, 
decreasing  automatically  for  turns  so  that  a  minimum  of  three  gates  is  displayed  on  the 
HMD  when  the  nearest  gate  is  centered.  This  technique  will  result  in  3-5  gates  being 
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displayed  at  any  given  time.  Should  the  position  of  the  nearest  gate  ever  exceed  the  FOV 
limitations  of  the  HMD,  the  pathway  symbology  should  be  replaced  by  flight-director 
symbology  until  the  nearest  gate  is  returned  to  the  HMD  FOV.  The  lateral  dimension  of 
the  nearest  gate  should  represent  the  equivalent  of  ±2  dots  of  lateral  deviation  on  a 
standard  Instrument  Landing  System  (ILS)  display  while  the  vertical  gate  dimension 
should  represent  ±2  dots  of  vertical  flight-path  deviation  on  the  ILS.  The  following  gates 
should  be  rendered  in  perspective,  and  be  occluded  by  other  HMD  symbology  when 
appropriate.  The  normal  color  of  the  gates  should  be  distinctive  from  other  HMD 
symbology,  with  black  being  recommended.  The  color  could  change  (e.g.,  to  yellow) 

100  ft  above  Decision  Height.  Based  on  an  appropriate  condition  for  the  specific  aircraft 
(i.e.,  a  substantial  increase  in  throttle,  speedbrake  retraction,  etc.),  the  pathway  should 
change  automatically  to  display  the  Missed  Approach  Procedure  path.  The  pathway 
symbology  should  disappear  automatically  at  Decision  Height  (unless  a  missed  approach 
is  performed). 

For  combat  operations,  such  as  low-level  penetration  of  enemy  airspace,  pathway-in-the- 
sky  guidance  displayed  on  a  HUD  has  also  shown  promise  in  research,  primarily  by 
improving  pilot  SA  (Reising  and  Snow,  1992).  For  this  purpose,  the  option  of  displaying 
pathway  symbology  on  the  HMD,  along  with  synthetic  virtual  terrain  and  ground 
features,  was  suggested.  This  symbology  should  correspond  to  that  shown  on  a  “God’s 
Eye”  view  provided  by  a  HDD  digital  moving-map  display.  Both  the  pathway  and  the 
virtual  terrain  should  be  earth  referenced  and  conformal  to  the  outside  world.  Whenever 
the  virtual  terrain  is  visible  in  the  HMD  FOV,  the  HMD  Horizon  Line  display,  described 
earlier,  should  be  eliminated  to  avoid  redundancy.  The  conformal  horizon  should  return 
to  the  display,  however,  whenever  the  virtual  terrain  lies  completely  outside  the  HMD 
FOV.  Pathway  gates  should  be  positioned  approximated  each  10  secs  of  flight  along  the 
planned  path.  There  was  some  interest  in  implementing  a  control  so  that  the  extent  of  the 
pathway  gates  in  this  HMD  mode  could  be  varied  by  the  pilot  on  a  real-time  basis,  so  that 
they  might  be  extended  or  retracted  along  the  future  planned  path  as  desired.  Further 
evaluation  is  required,  however,  to  determine  the  usefulness  of  this  feature.  The  color  of 
the  pathway  gates  should  be  a  distinctive  color  so  that  they  can  be  easily  distinguished 
from  other  symbology,  but  should  change  color  to  red  whenever  they  are  shown  to  enter 
the  lethal  envelope  of  threat  weapons.  The  default  lateral  dimension  of  the  pathway  gates 
in  this  mode  should  represent  ±500  ft  from  path  centerline,  while  the  vertical  dimension 
should  be  ±150  ft  from  the  center  of  the  gate.  These  dimensions  should  be  programmable 
during  preflight.  No  objection  was  made  to  the  symbology  currently  being  used  for 
Terrain-Following  Radar  (TFR)  guidance,  which  can  be  superimposed  along  with  the 
pathway  symbology  on  the  VHUD.  The  pathway  symbology  should  be  easily  decluttered 
when  desired. 

Experience  in  flying  with  pathway-in-the-sky  symbology  has  shown  that  it  is  sometimes 
difficult  to  predict  the  optimum  flight  path  for  penetrating  the  next  gate  near  the  center, 
particularly  when  in  a  turn.  One  suggested  method  of  facilitating  this  task  is  to  permit 
display  of  flight-director  symbology  at  all  times  to  aid  in  directing  the  pilot  along  the 
desired  trajectory.  Although  effective,  this  approach  also  adds  significant  visual  clutter. 
The  preferred  approach  suggested  by  this  study  is  to  mechanize  the  pathway  so  that  the 
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nearest  gate  is  always  positioned  at  the  aircraft’s  current  position  along  the  desired  track 
(i.e.,  stationary  with  regard  to  the  aircraft’s  position),  while  the  remaining  gates  appear  to 
approach  the  own-aircraft  reference  symbol  (i.e.,  CDM,  FPM)  until  they  “merge”  with 
the  nearest  gate.  With  this  mechanization  the  pilot  can  be  assured  that  his  aircraft  is 
positioned  properly  along  the  desired  trajectory,  even  between  gates,  as  long  as  the  own- 
aircraft  reference  symbol  is  positioned  correctly  with  reference  to  the  nearest  gate. 

As  an  option  to  the  pathway-in-the-sky  symbology  described  above,  a  “black  Line” 
course  display  should  be  provided.  The  actual  color  of  this  line  should  be  identical  to 
that  of  the  pathway  gates,  and  chosen  for  its  distinctiveness  and  visibility  against  all 
backgrounds.  This  display  should  be  earth  referenced  and  appear  to  lie  along  the 
displayed  virtual  terrain  corresponding  to  the  planned  track.  It  was  thought  that  these 
concepts  offer  a  reasonable  tradeoff  for  flight-path  guidance  with  a  minimum  of  visual 
clutter. 


FIGURE  11:  HMD  WITH  VIRTUAL  TERRAIN  AND  PATHWAY-IN-THE-SKY 
(HMD  LOS  20°  Left  of  Aircraft  Nose,  Showing  VHUD) 


Figure  1 1  illustrates  some  of  these  features.  In  this  depiction  the  pilot’s  LOS  (Aiming 
Reference)  is  about  20°  left  of  the  aircraft’s  nose  and  10°  above  the  horizon.  As  this  LOS 
is  within  the  assumed  boundary  for  displaying  the  VHUD,  that  display  is  also  visible. 

The  Aiming  Reference  and  the  HMD  LOS  data  block  (upper  left-hand  comer  of  HMD) 
are  head  referenced,  moving  with  the  pilot’s  head.  The  virtual  terrain,  FPM,  pathway-in- 
the-sky,  and  projected  ground  track  are  earth  referenced,  appearing  at  the  point  in  space 
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where  the  features  they  represent  would  actually  be  if  they  could  be  seen.  The  remaining 
data  elements  on  the  VHUD  are  aircraft  referenced,  their  positions  remaining  fixed  in 
relation  to  the  aircraft  (i.e.,  on  the  VHUD). 


6.16  GROUND  FEATURES,  WAYPOINTS,  DATALINK  SYMBOLOGY 

6.16.1  FUNCTIONALITY 

The  recent  confluence  of  technologies,  including  the  Global  Positioning  System  (GPS), 
the  availability  of  digital  terrain  data,  accessibility  to  real-time  data  via  datalink,  and 
advanced  HMD  display  capabilities  afford  the  opportunity  to  provide  the  pilot  with  a 
virtual  view  of  the  outside  world  to  either  replace  or  supplement  the  view  provided  by  the 
naked  eye  or  onboard  electronic  sensors.  The  SMEs  in  this  study  believe  this  capability 
could  be  extremely  valuable  for  improving  SA  and  facilitating  mission  success, 
especially  under  conditions  of  reduced  visibility. 

6.16.2  FORMAT  AND  MECHANIZATION 

Since  the  display  of  synthetic  terrain,  cultural  features,  navigational  waypoints,  and 
sensor  and  datalink  entities  on  the  HMD  have  great  potential  for  increasing  visual  clutter, 
several  levels  of  selectability/declutter  are  required.  The  lowest  level  of  display  would 
include  only  navigational  waypoints  and  targeting  symbology  generated  by  own  ship.  A 
second  level  would  include  off-board  datalink  entities,  while  a  logical  third  display  level 
would  add  programmed  cultural  or  tactical  features,  like  political  boundaries, 
FEBA/FLOT,  etc.  It  was  suggested  that  a  “New  Guy”  audio/voice  alert  be  provided 
whenever  a  datalink  entity  is  initially  made  available  for  display,  in  order  to  call  the 
pilot’s  attention  to  this  entity.  A  “wire-mesh”  virtual  terrain  would  represent  the  next 
higher  level  of  display,  with  the  granularity  of  the  mesh  preferably  selectable  by  the  pilot, 
at  least  during  preflight.  Finally,  “solidity,”  color,  and  possibly  other  textural  features 
can  be  added  to  the  display.  All  these  features  would,  of  course,  be  conformal  to  the 
outside  world.  Such  a  display  would  allow  the  pilot  to  fly  in  his  own  virtual  world, 
enhanced  by  the  addition  of  digital  entities  and  other  cultural  features  to  provide 
expanded  SA  under  any  visibility  conditions.  Considering  the  visual  clutter,  the 
usefulness  of  “solid”  virtual  terrain  requires  further  study. 

A  suggested  feature  is  to  provide  the  option  of  rendering  the  more  solid  virtual  elements, 
such  as  solid  virtual  terrain,  transparent  in  areas  of  the  HMD  that  offer  “see-through” 
imagery.  For  instance,  solid  virtual  terrain  could  revert  automatically  to  wire  mesh  and 
solid  ground  features  could  change  to  outlines  only  when  they  are  drawn  in  see-through 
regions  of  the  HMD,  allowing  the  pilot  to  detect  actual  objects  either  visually  or  electro- 
optically  when  visibility  conditions  permit.  This  concept  is  illustrated  in  Figure  12. 
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FIGURE  12:  VHUD  WITH  TRANSPARENT  VIRTUAL  TERRAIN 


Liberal  use  of  color  in  this  display  is  important  for  intuitiveness  and  clutter  reduction.  No 
objection  was  made  to  the  symbology  and  color  guidance  provided  by  MIL-STD-1787C. 


6.17  SENSOR  IMAGERY 
6.17.1  FUNCTIONALITY 

Modem  tactical  aircraft  are  typically  equipped  with  an  array  of  onboard  sensors  to  extend 
and  amplify  the  pilot’s  vision,  providing  enhanced  ability  to  locate,  identify,  and  destroy 
targets  of  interest,  maintain  SA,  and  navigate  under  difficult  visual  conditions.  These 
systems  include  Forward-Looking  Infrared  (FLIR),  AGM-65  MAVERICK  video,  Night 
Vision  Goggles  (NVGs),  Television  Camera  System  (TCS),  Synthetic-Aperture  Radar 
(SAR),  etc.  Increasingly,  these  onboard  sensors  will  be  augmented  by  off-board  sensor 
capability  via  datalink.  Displays  for  these  systems  must  be  provided  and  controlled,  and 
integrated  with  the  HMD  for  optimal  employment.  This  may  involve  the  display  of  video 
from  multiple  sensors  simultaneously.  Each  of  these  sensors  will  likely  have  gimbal  or 
masking  limits  that  require  monitoring.  In  addition,  when  the  sensors  involved  are 
weapon  seekers,  it  may  be  advantageous  to  prioritize  multiple  weapon-target  pairings  for 
later  engagement. 
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6.17.2  FORMAT  AND  MECHANIZATION 


The  ability  to  project  imagery  on  the  HMD  is  highly  dependent  on  the  mechanization  of 
the  HMD  itself.  As  discussed  above,  the  baseline  hardware  assumed  for  this  study  is  that 
of  Strike  Helmet  21.  As  Strike  Helmet  21  is  a  developmental  program,  the  technical 
hardware  specifications  and  capabilities  are  not  well  established.  Early  in  this  program 
our  team  was  provided  with  some  general  specifications,  which  later  proved  to  be 
inaccurate.  Although  revised  specifications,  limitations,  and  characteristics  were 
provided  later  in  the  program,  it  was  too  late  to  take  these  fully  into  consideration. 
Besides,  there  is  still  some  uncertainty  and  doubt  that  even  these  will  ultimately  prove  to 
be  the  final  parameters.  Therefore,  our  team  chose  to  offer  only  general 
recommendations  that  could  be  applicable  to  a  range  of  HMD  capabilities. 

In  general,  it  is  understood  that  Strike  Helmet  21  has  NVG  capability  by  displaying  the 
output  of  light  intensifier  tubes.  In  addition,  synthetic  video  can  be  displayed  over  at 
least  portions  of  the  HMD  FOV  by  four  Active  Matrix  Organic  Light  Emitting  Diodes 
(AMOLEDs).  Apparently,  only  the  center,  binocular  third  of  the  FOV  can  display 
transparent  imagery,  and  the  pilot  can  choose  to  display  synthetic  video  on  only  two 
AMOLEDs  at  a  given  time.  Due  to  these  limitations,  some  of  the  recommendations  of 
this  study  may  be  impractical  to  implement  using  Strike  Helmet  21. 

A  head-steered  FLIR  has  demonstrated  substantial  performance  advantages  in  simulation 
over  a  FLIR  system  slewed  manually,  or  fixed  to  the  aircraft  boresight,  and  not  correlated 
with  the  pilot’s  head  movement  (Boucek  &  Hassoun,  1996).  These  performance 
advantages  were  evidenced  in  improved  location  and  targeting  of  ground  targets,  and 
increased  pilot  SA.  Osgood,  Wells,  and  Meador  (1995)  also  demonstrated  improved 
performance  with  a  head-steered  FLIR  in  a  MAVERICK  missile  targeting  task. 
Considering  these  results,  the  option  of  projecting  head-steered  onboard  sensor  video  on 
the  HMD,  as  well  as  a  HDD,  should  be  provided.  Head-steered  onboard  sensor  displays 
should  be  head-referenced  on  the  HMD  so  that  the  pilot’s  view  corresponds  to  his  visual 
LOS  and  head  orientation.  Large  video  insets  can  be  exceptionally  compelling,  and  there 
will  be  a  tendency  for  the  pilot  to  attempt  to  fly  the  aircraft  by  reference  to  this  video. 
Disorientation  is  very  likely  unless  the  video  inset  matches  the  pilot’s  head  in  both 
azimuth  and  roll  orientation. 

Boucek  &  Hassoun  also  showed  that  there  was  little  advantage  to  be  gained  in  targeting 
by  the  use  of  targeting  FLIR  video  insets  on  the  HMD,  rather  than  on  a  conventional 
HDD.  Targeting  performance  was  actually  superior  using  the  HDD  for  targeting, 
although  subjective  pilot  ratings  were  higher  for  the  inset  HMD  targeting  FLIR.  Osgood, 
et.  al.  showed  performance  improvement,  both  objectively  and  subjectively,  using  inset 
MAVERICK  video  on  the  HMD.  Implementation  issues  such  as  display  size  and 
resolution  will  likely  be  important  factors  in  determining  the  preferred  approach.  These 
factors  being  equal,  it  is  likely  that  some  performance  and  SA  improvements  may  be 
achieved  by  the  inset  targeting  video,  so  it  is  recommended  that  this  option  be  offered.  In 
order  to  reduce  the  potential  for  spatial  disorientation,  however,  it  is  recommended  that 
ONLY  sensor  video  correlated  with  the  pilot’s  head  orientation  be  displayed  on  the  HMD 
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unless  this  display  covers  no  more  than  about  1/16  of  the  total  HMD  FOV  area. 

Likewise,  the  same  limitation  applies  to  SAR  imagery  because  of  its  non-real-time 
nature.  Therefore,  once  the  target  is  locked  and  the  video  image  no  longer  reflects  real¬ 
time  motion  correlated  with  the  pilot’s  head,  the  video  inset  should  be  reduced  in  size  and 
become  earth-referenced,  so  that  it  is  only  visible  when  within  the  HMD  FOV.  When  a 
locked  target  lies  outside  the  HMD  FOV  a  reflective  target  locator  line  (TT  .T  .)  described 
in  greater  detail  in  Section  6.18.2  below,  should  appear  to  allow  the  pilot  to  stay  oriented 
with  the  target’s  position  and  facilitate  visual  reacquisition.  Concurrently,  the  target 
video  inset  image  should  be  moved  to  the  extreme  lower  left  of  the  HMD  FOV,  since  this 
area  has  been  shown  to  attract  the  viewer’s  attention  least  (Previc,  2000).  The  hypothesis 
here  is  that  the  less  attention-grabbing”  the  inset,  the  less  potential  for  causing  spatial 
disorientation.  Alternatively,  the  image  could  be  moved  to  a  HDD.  Retaining  the  video 
inset  image  of  the  target  in  either  the  HMD  or  HDD  when  the  target  is  outside  the  HMD 
FOV  allows  the  pilot  to  monitor  the  lock  status  even  when  the  target  is  not  normally  in 
view. 

The  SMEs  in  this  study  also  desired  the  ability  to  assemble  a  “shoot  list”  of  multiple 
targets  for  subsequent  attack  in  a  defined  sequential  order.  For  instance,  each  onboard 
weapon  of  a  given  type,  such  as  MAVERICK,  might  be  locked  to  a  separate  target. 

Before  lock,  the  selected  weapon  might  be  manually  slewed  or  head  slewed  to  the  desired 
target,  with  its  video  continuously  displayed  either  on  a  HDD  or  HMD  inset.  As  long  as 
the  weapon  is  being  head-steered,  its  video  might  be  enlarged  on  the  HMD  for  better* 
resolution  and  target  discrimination.  This  inset  might  occupy  much  of  the  center  section 
of  the  Strike  Helmet  21  FOV,  for  instance.  Once  locked  to  its  target,  however,  the 
weapon  video  would  be  reduced  in  size  and  become  earth-stabilized.  It  would  also 
automatically  be  assigned  the  next  available  priority  number,  and  its  inset  video  so 
labeled.  The  pilot  would  then  be  free  to  select  the  next  weapon  and  lock  another  target, 
which  would  be  assigned  a  sequential  priority  number.  As  long  as  targets  remain  located 
within  the  HMD  FOV,  their  video  insets  would  remain  earth-stabilized.  If  any 
designated  target  leaves  the  HMD  FOV,  its  video  would  be  placed  either  along  the  left- 
hand  side  of  the  HMD,  stacked  in  priority  order  from  bottom  to  top,  or  removed  to  the 
HDD  according  to  the  pilot  s  selection.  Only  the  “Priority  1”  target  would  be  assigned  a 
TLL,  indicating  the  LOS  to  the  target  and  the  number  of  degrees  off  HMD  boresight. 

The  selected  weapon  automatically  defines  the  Priority  1  target.  Whenever  a  target  in  the 
stack  is  subsequently  located  within  the  HMD  FOV,  the  appropriate  target  video  inset 
and  associated  priority  number  would  leave  the  stack  and  become  earth  stabilized  in  its 
appropriate  position.  The  designated  Priority  1  target  should  be  clearly  discriminated 
visually  among  the  video  insets  on  either  the  HMD  or  HDD.  This  might  be  accomplished 
by  highlighting  the  priority  inset  in  a  red  border. 

Typically,  each  onboard  sensor  will  have  some  defined  gimbal  or  masking  limit.  The 
pilot  needs  some  guidance  regarding  the  location  of  these  limits,  or  warning  that  the 
limits  are  being  approached,  with  sufficient  lead  time  and  guidance  to  avoid  exceeding 
those  limits.  A  suggested  approach  for  this  function  would  be  to  provide  an  audio/voice 
cue  (i.e.,  “Masking  Sensor”)  to  alert  the  pilot  when  within  15°  of  the  limit.  The  border 
around  the  sensor  image,  or  its  inset,  could  then  be  used  to  inform  the  pilot  of  the  limit 


42 


being  approached.  For  instance,  if  the  left-hand  gimbal  limit  of  the  MAVERICK  locked 
to  the  Priority  3  target  is  being  approached,  the  left-hand  border  of  that  video  inset  might 
turn  red  and  blink  at  a  5-Hz  rate  when  within  15°  of  the  gimbal  limit,  and  become  steady 
red  5°  from  the  limit,  until  the  condition  is  corrected  (presumably  by  turning  to  the  left). 
When  the  sensor  approaching  gimbal  limits  is  associated  with  the  Priority  1  target,  the 
TLL  and  reflective  cue  should  also  be  blinked.  These  concepts  are  illustrated  in  Figure 
13. 


HMD  LOS 


TLL  with 
Reflective 
Cue 


HMD  FOV 


Conformal 

Horizon 


H  --Targeting  FUR 
H  NFOV  Boundary 


£  ...  r  -  .rr;|^  l^ruv  DUUIIUdiy 

■  m  m.  m  «  ••  _■  y ^y.-. --  :,v .  ZJ  . 

Head-Steered  % 


MAVERICK  Vide 
■r;-\  Stack  with 
Targeting  Priority 


V 


ri-,  Targeting  FLIR 

Numbers  WFOV  Inset 

r-.rV/.v.-  ■  :  .  /  ___  ;-y  :-;7-;y-  ■;  V  ;  ■, 


'MAVERICK  Video 
Inset  Stack 


Location  of 
Priority  3  Target 


Left  MAVERICK  Gimbal 
Limit  Warning 


FIGURE  13:  HMD  WITH  INSET  FLIR  AND  WEAPON  VIDEO 


6.18  RADAR/THREAT  WARNING  SYSTEM 
6.18.1  FUNCTIONALITY 

Modem  tactical  aircraft  are  equipped  with  various  systems  for  electronically  detecting  the 
presence  and  threat  level  of  an  array  of  threat  sensors  and  weapons.  Traditionally  the 
output  of  these  systems  has  been  depicted  on  separate  HDDs.  The  SMEs  in  this  study 
believe  that  there  are  several  advantages  to  integrating  these  displays  with  the  HMD. 

With  HMD  display,  critical  information  from  these  systems  would  be  available  to  the 
pilot  without  forcing  his  head  back  into  the  cockpit.  In  addition,  the  ability  to  align  the 
HMD  with  the  threat  LOS  should  greatly  improve  the  pilot’s  ability  to  acquire  threats 
visually,  which  in  some  cases  greatly  improves  the  chances  of  evasion  and  survival. 
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6.18.2  FORMAT  AND  MECHANIZATION 

This  study  recommends  that  threats  detected  by  a  radar  warning  receiver  (RWR)  or  other 
threat  warning  system  (TWS)  be  depicted  within  the  HMD  or  VHUD  FOV  using  system- 
specific  symbols  that  denote  the  type  of  threat  detected  (i.e.,  SAM,  Fighter,  Search  Radar, 
etc.),  along  with  appropriate  alphanumerics  immediately  above  the  symbol  to  specify 
threat  type  (i.e.,  SA-6,  Su-27,  UNK,  etc.).  There  were  no  objections  to  the  symbology 
and  color  conventions  provided  in  MIL-STD-1787C  for  this  purpose,  although  symbols 
displayed  on  the  HMD  should  be  outline-only  to  reduce  visual  clutter.  This  symbology 
should  be  earth  referenced  according  to  the  position  of  the  threat  in  space.  Additional 
alphanumerics  should  provide  threat  priority  (below),  and  range  to  the  threat  in  NM  to 
the  tenth  of  a  NM  if  that  level  of  range  accuracy  is  available  (to  the  right  of  the  symbol). 
Figure  14a  illustrates  an  example. 
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FIGURE  14a:  SAMPLE  THREAT  WARNING  SYMBOLOGY 
(SA-6  SAM,  Threat  Priority  1,  Range  27.2  NM) 


When  the  threat  LOS  places  it  outside  the  FOV  of  the  HMD  or  VHUD,  a  target  locator 
line  (TLL)  should  emanate  from  the  HMD  Aiming  Reference  (or  fixed  VHUD  Aiming 
Reference  if  displayed)  to  the  LOS  of  the  threat,  extending  to  the  FOV  limit  of  the  HMD. 
The  color  of  the  target  locator  line  should  correspond  to  the  threat  level,  as  suggested 
above,  and  the  threat  symbol  and  associated  alphanumerics  should  form  a  reflective  cue 
attached  to  the  TLL.  Reflective  TLL  mechanization  has  been  shown  in  research  to  offer 
advantages  in  visual  acquisition  of  off-boresight  targets,  and  is  the  mechanization 
preferred  by  most  pilots  participating  in  this  research  (Fechtig,  Boucek,  and  Geiselman, 
1998;  Craig,  Marshall,  and  Jordan,  1997;  Geiselman,  1999). 

In  the  reflective  mechanization  approach,  the  reflective  cue  moves  along  the  TLL,  with 
its  distance  from  the  OUTER  end  of  the  TLL  representing  the  angular  difference  between 
the  HMD  boresight  and  the  target  LOS.  When  the  target  is  far  off  the  HMD  boresight  the 
reflective  symbology  is  located  near  the  inboard  end  of  the  TLL.  As  the  HMD  boresight 
and  the  target  LOS  converge,  the  symbology  slides  out  the  TLL  toward  the  edge  of  the 
HMD  FOV,  disappearing  just  as  the  symbology  merges  with  the  actual  target  LOS  at  the 
edge  of  the  HMD  FOV.  This  study  recommends  that  the  TLL  always  extend  to  the  limit 
of  any  transparent  portion  of  the  HMD  FOV.  Assuming  this  FOV  is  not  circular,  this 
means  the  length  of  the  TLL  will  change  depending  on  the  orientation  of  the  TLL  with 
respect  to  the  HMD  FOV .  The  distance  of  the  target  reference  from  the  outer  end  of  the 
TLL,  however,  should  continue  to  represent  the  angular  distance  to  the  target  LOS  from 
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the  adjoining  edge  of  the  HMD  FOV.  Degrees  from  the  HMD  Aiming  Reference  to  the 
threat  LOS  should  be  displayed  digitally  to  the  side  of  the  TLL  near  the  HMD  Aiming 
Reference. 

The  TLL  symbology  format  is  recommended  in  this  report  for  several  applications, 
including  active  own-ship  sensor  tracks,  passive  sensor  tracks,  weapon  seeker  tracks,  etc. 
With  multiple  sensors  and  weapons,  and  sensors  capable  of  tracking/detecting  multiple 
targets  at  once,  priorities  need  to  be  established  for  TLL  display.  It  has  been  shown  by 
both  operational  and  research  experience  that  only  one  TLL  should  be  presented  on  the 
HMD  at  any  given  time  (Geiselman,  1999).  For  a  given  weapon  or  sensor  type,  TLL 
priority  is  established  as  the  highest  priority  target,  determined  either  automatically  by 
the  system  itself  or  by  the  pilot  when  that  capability  is  available.  When  TLLs  are 
indicated  for  multiple  purposes  or  systems,  such  as  an  onboard  weapon  targeting  sensor 
and  RWR/TWS,  some  prioritization  scheme  needs  to  be  devised.  Such  a  prioritization 
scheme  may  be  either  manual  or  automated.  Our  SMEs  favored  a  manual  prioritization 
scheme,  in  which  the  pilot  would  select  either  OFFENSIVE  or  DEFENSIVE  priority  for 
the  TLL.  OFFENSIVE  priority  would  favor  a  targeting  TLL  over  an  RWR/TWS  TLL, 
and  vice  versa.  Priority  selection  could  be  either  by  HOTAS  or  DVI  control.  If  an 
automated  scheme  is  provided,  this  study  recommends  that  onboard  targeting  sensors  be 
given  TLL  priority  over  low-  and  medium-threat  RWR/TWS  contacts.  High-threat 
airborne  RWR/TWS  contacts  (i.e.,  fighters,  missiles,  etc.)  should  assume  TLL  priority. 
The  most  effective  prioritization  approach  should  be  the  subject  of  further  study.  In 
either  the  manual  or  automated  case,  whenever  a  TLL  is  indicated  for  RWR/TWS 
display,  but  is  suppressed  by  a  higher  priority  TLL,  the  LOS  to  the  RWR/TWS  contact 
might  be  represented  by  a  short  arrow  at  the  edge  of  the  HMD  pointing  toward  the  threat 
LOS,  along  with  the  appropriate  threat  symbology  and  alphanumerics  described  above. 
This  approach,  illustrated  in  Figure  14b,  allows  the  pilot  to  continue  to  monitor  the 
developing  threat  situation  while  attending  to  other  tasks  as  appropriate. 

Current  RWR/TWS  systems  typically  employ  sound  to  augment  their  visual  threat 
displays,  initially  in  the  form  of  distinctive  audio  tones  representing  various  threat 
systems,  and  more  recently  by  voice  cues.  Although  results  were  mixed  in  early  systems, 
especially  in  threat-rich  environments  when  some  pilots  actually  chose  to  turn  off  the 
RWR  rather  than  cope  with  the  cacophony  of  a  multitude  of  “beeps  and  squeaks,”  the 
participants  in  this  study  believe  that  the  appropriate  use  of  sound  is  an  asset  in  this 
application.  A  single  voice  warning  on  first  detection  of  each  new  threat  is  preferred  over 
continuous  artificial  warning  tones  or  “new-guy”  audio.  Although  the  results  of  research 
into  the  effectiveness  of  three-dimensional  (3-D)  sound  in  improving  threat  acquisition 
has  been  mixed  (Boucek  &  Hassoun,  1996),  the  participants  in  this  study  consider  this 
technique  to  be  potentially  valuable,  at  least  for  initial  SA  of  threat  location.  The  HMD 
TLL  mechanization  recommended  above  is  expected  to  provide  the  most  effective 
method  of  target  visual  acquisition. 
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FIGURE  14b:  TWS  SYMBOLOGY 

(Threat  outside  HMD  FOV,  but  TLL  Suppressed  by  Higher  Priority  Application) 


When  the  TWS  actually  detects  a  missile  inbound  to  Own  Ship,  there  should  be  a 
distinctive  voice/audio  cue  to  this  event.  TLL  priority  should  be  assumed  by  this  threat, 
and  the  threat  symbology  depicted  should  revert  to  a  standard,  easily  interpreted 
“incoming  missile”  symbol,  as  illustrated  in  Figure  14c. 
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FIGURE  14c:  INCOMING  MISSILE  SYMBOLOGY 
(SA-6  Inbound,  Range  12.4  NM,  65°  Off  HMD  LOS) 
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6.19  ELECTRONIC  /  INFRARED  COUNTERMEASURES  (ECM  /  IRCM) 
SYSTEM  /  EXPENDABLES 

6.19.1  FUNCTIONALITY 

Modem  tactical  aircraft  are  typically  equipped  with  an  array  of  ECM  and  IRCM  jammers 
and  countermeasures  expendables.  It  is  often  valuable  to  know  the  operational  status  of 
jamming  systems,  including  whether  they  are  actively  jamming  and,  if  so,  in  which 
mode(s).  Similarly,  the  pilot  should  be  aware  of  when  expendables,  such  as  chaff  and 
flares,  are  being  dispensed  by  automatic  systems,  and  that  they  are  deploying  properly  in 
response  to  manual  activation.  In  addition,  since  there  is  always  a  finite  supply  of 
expendables,  it  is  important  for  the  pilot  to  be  aware  of  his  remaining  supply,  since  this 
knowledge  may  impact  his  tactical  decisions. 

6.19.2  FORMAT  AND  MECHANIZATION 

Although  the  SMEs  in  this  study  recognized  the  critical  nature  of  these  functions,  they 
suggested  that  the  preferred  location  for  providing  the  remaining  quantity  of  expendables 
is  the  HDD.  Likewise,  audio  /  voice  cues  were  recommended  for  ECM  /  IRCM 
operational  status  information,  in  addition  to  the  HDD,  to  avoid  increasing  visual  clutter 
in  the  HMD  or  VHUD. 


6.20  SENSOR  AUTO-ACQUISITION  GUIDANCE 
6.20.1  FUNCTIONALITY 

Modem  tactical  aircraft  are  typically  equipped  with  tracking  sensors,  such  as  radar  and 
guided-missile  seekers.  Often  these  sensor  systems  provide  automated  means  of 
achieving  a  fully  automatic  track  mode.  With  the  advent  of  digital  aircraft  systems  and 
HUDs,  common  HUD-based  “auto-acquisition”  modes  include  “boresight,”  which 
automatically  “locks”  any  target  detected  near  the  aircraft’s  FRL.  The  F-15  provides  a 
“Auto-Guns”  mode  that  greatly  expands  the  boresight  auto-acquisition  lateral  and  vertical 
limits  to  roughly  the  bounds  of  the  HUD  FOV.  Another  popular  auto-acquisition  mode  is 
“Vertical  Scan  Lock,”  (VSL)  in  which  the  sensor  scans  the  aircraft’s  vertical  plane  within 
tight  lateral  limits,  from  roughly  from  the  FRL  to  the  sensor  vertical  gimbal  limits.  A 
number  of  other  auto-acquisition  techniques  have  also  been  fielded. 

It  has  been  stated  that  the  primary  function  of  the  HMD  is  to  aim  weapons  and  onboard 
sensors,  therefore  it  stands  to  reason  that  the  HMD  should  be  the  principal  means  by 
which  sensor  and  weapon  auto-acquisition  modes  should  be  controlled.  In  fact,  the  HMD 
may  be  the  ONLY  means  of  performing  this  function  automatically.  If  that  is  the  case, 
the  HMD  Aiming  Reference  symbology,  as  discussed  above,  including  the  auto¬ 
acquisition  boundaries  of  the  sensor  selected,  will  suffice  to  provide  this  functionality. 
The  SMEs  in  this  study,  however,  strongly  believe  that  additional  auto-acquisition  modes 
should  still  be  provided. 
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6.20.2  FORMAT  AND  MECHANIZATION 

Suggested  auto-acquisition  modes  include  HMD  Boresight,  in  which  the  target  located 
within  a  few  degrees  of  the  HMD  Aiming  Reference  within  a  specified  range  will  be 
locked  automatically.  Such  a  mode,  with  very  restrictive  angular  boundaries,  would 
likely  provide  the  quickest  means  of  locking  a  visual  target.  Similarly,  an  HMD  Super- 
Search  mode  would  automatically  lock  any  target  within  the  HMD  FOV.  This  would 
enable  a  relatively  quick  means  of  locking  a  target  when  the  pilot  has  only  a  general  clue 
regarding  target  LOS.  Another  suggestion  might  be  called  “HMD  VSL,”  with  both 
WIDE  and  NARROW  options.  The  WIDE  option  would  scan  the  width  of  the  HMD 
FOV  in  azimuth,  and  from  the  bottom  of  the  HMD  FOV  to  the  vertical  limits  of  the 
sensor  vertically.  The  NARROW  option  would  scan  a  much  tighter  range  of  azimuth,  on 
the  order  of  ±10°,  to  provide  a  quicker  lock  on  a  target  with  a  known  location. 

Most  current  tracking  sensors  on  fighter  aircraft  have  gimbal  limits  on  the  order  of  60°- 
70°  in  azimuth  and  elevation.  With  these  fairly  restrictive  limits,  control  of  auto¬ 
acquisition  modes  via  the  HMD  may  be  adequate,  since  the  pilot  can  normally  adjust  his 
head  position  to  manipulate  the  auto-acquisition  scan  pattern  adequately.  If  sensors  are 
envisioned  with  greatly  expanded  gimbal  limits,  however,  additional  aircraft-based 
slewable  modes  may  be  necessary.  Even  with  current  sensors,  there  may  be  times,  as 
when  under  high  G,  when  the  pilot  does  not  have  sufficient  control  over  his  head  position 
to  allow  adequate  control  for  an  HMD-based  auto-acquisition  system.  This  problem  may 
be  alleviated  somewhat  by  the  fact  that  an  HMD-controlled  auto-acquisition  system  like 
the  HMD  VSL  mode  described  above  can  be  controlled  by  both  head  movement, 
including  pitch,  yaw,  and  roll  orientation,  and  aircraft  maneuvers,  especially  the 
adjustment  of  roll  attitude.  Still,  the  pilot  may  wish  to  keep  his  head  in  the  cockpit  for 
some  reason,  or  attend  to  another  direction,  during  this  operation.  Therefore,  there  may 
be  some  value  to  providing  an  aircraft-based  auto-acquisition  system  also,  similar  to 
those  currently  operational.  Further  research  is  warranted  in  this  area. 

Whenever  an  auto-acquisition  mode  is  selected,  it  is  of  great  value  to  the  pilot  to  have  an 
indication  of  the  extent  of  the  scan  pattern.  Figure  15  depicts  a  suggested  approach  for 
depicting  the  lateral  extent  of  the  HMD  VSL  (NARROW  option). 
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HMD  FOV 


FIGURE  15:  HMD  VSL  (NARROW)  SYMBOLOGY 


6.21  AIR-TO-AIR  WEAPONS  SYMBOLOGY 

6.21.1  FUNCTIONALITY 

In  order  to  deploy  air-to-air  weapons  effectively  the  pilot  requires  a  great  deal  of 
information.  This  includes  details  of  the  capabilities  of  the  weapon  selected  (i.e., 
maximum  kinematic  range,  maximum  relative  range,  maximum  “tum-and-run”  range, 
minimum  range,  etc.);  optimum  firing  attitude;  target  range,  aspect,  closure,  and 
maneuvers;  effective  range  of  threat  air-to-air  weapons,  etc.  Providing  all  this 
functionality  in  a  usable,  intuitive  form  is  a  challenge,  but  the  HMD  is  the  logical  place 
for  it  in  many  circumstances. 

6.21.2  FORMAT  AND  MECHANIZATION 

The  baseline  format  recommended  by  this  study  is  the  Normalized  In-Range  Display 
(NIRD),  described  in  Geiselman,  1998b.  This  format  provides  color-  and  shape-coded 
missile  parameters  to  the  pilot.  The  NIRD  maps  a  “normalized”  Dynamic  Launch  Zone 
(DLZ)  display  to  an  Allowable  Steering  Error  (ASE)  circle.  In  addition,  color-coding  has 
been  adopted  from  the  “Red  Means  Shoot”  color-coding  scheme  shown  to  be  most 
effective  in  this  application  (Geiselman,  Post,  Brickman,  Rogers-Adams,  Hettinger,  & 
Haas,  1998). 

Also  included  in  the  NIRD  are  target  range  and  closure,  target  aspect  angle,  and 
collision/aim  guidance.  The  diameter  of  the  ASE  circle  may  either  be  fixed,  or  varied  to 
provide  a  cue  to  the  type  of  weapon  selected.  For  instance,  a  smaller  ASE  circle  might 
represent  a  SIDEWINDER  missile,  while  a  somewhat  larger  circle  would  indicate  an 
AMRAAM.  Whichever  technique  is  employed,  however,  the  size  of  the  ASE  circle 
remains  fixed  for  a  given  weapon.  As  in  the  current  convention,  the  Aim  Dot  provides  a 
cue  to  the  pilot  for  achieving  either  an  optimum  collision  course  with  the  designated 
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target  or  the  optimum  pitch  and  heading  for  launching  the  weapon  at  a  given  instant.  It  is 
suggested  that  when  target  range  is  greater  than  1.1-times  the  maximum  aerodynamic 
range  (RMAX2)  of  the  selected  weapon,  the  Aim  Dot  represents  collision  course. 
Therefore,  when  the  Aim  Dot  is  located  in  the  center  of  the  ASE  circle,  Own  Ship  is  on  a 
collision  course  with  the  target.  At  ranges  between  1.1-times  RMAX2  and  RMAX2,  the 
Aim  Dot  transitions  from  a  collision-steering  dot”  to  a  weapon-aiming  reference.  At 
ranges  inside  RMAX2  the  Air  Dot  should  function  as  an  aiming  reference,  so  that  placing 
the  Aim  Dot  in  the  center  of  the  ASE  circle  provides  the  optimum  Own-Ship  launch 
attitude  for  given  conditions.  Movement  of  the  Aim  Dot  should  be  compressed  so  that  its 
location  inside  the  ASE  circle  indicates  acceptable  (although  possibly  not  optimal)  Own- 
Ship  course  and  pitch  for  weapon  launch. 

Target  range  is  presented  in  analog  format  by  the  position  of  the  Range  Index  in  relation 
to  the  ASE  circle.  The  Range  Index  is  a  small  equilateral  triangle  with  one  point 
touching  the  outside  edge  of  the  ASE  circle;  its  position  on  the  periphery  of  the  ASE 
circle  represents  target  range  as  related  to  the  performance  parameters  of  the  selected 
weapon.  Clockwise  movement  of  the  Range  Index  indicates  increasing  range  and 
counter-clockwise  motion  indicates  decreasing  range.  The  12:00  position  represents  both 
zero  range  and  all  ranges  equal  to,  or  greater  than  1.33-times  RMAX2.  Target  closure  (in 
kts)  is  provided  digitally  just  outside  the  Range  Index  radially  in  relation  to  the  center  of 
the  ASE  circle. 


The  NIRD/ASE  circle  depicts  weapon  parameters  by  the  use  of  position,  color,  and  shape 
coding.  In  the  event  that  the  color  capability  of  the  HMD  is  damaged  or  unavailable,  all 
necessary  symbology  will  be  available  for  use  by  the  pilot  due  to  the  redundant  shape  and 
position  coding.  Suggested  position  and  color  coding  is: 

^  12:00  clockwise  to  3:00,  with  3:00  representing  Minimum  Range  (RMIN)  -  Greer- 

>  3:00  clockwise  to  4:30,  with  4:30  representing  Turn  &  Run  Range  (RTR)  -  Double 
Thick  red 

>  4:30  clockwise  to  6:00,  with  6:00  representing  RMAX1  -  Thick  Red 

>  6:00  clockwise  to  9:00  with  9:00  representing  Maximum  Aerodynamic  Range 
(RMAX2)  -  Red 

>  9:00  clockwise  to  12:00,  representing  beyond  RMAX2  - 

Any  portion  of  the  NIRD/ASE  circle  representing  ranges  BEYOND  the  target  should  be 
drawn  using  dashed  lines.  Everything  between  the  target  and  Own  Ship  is  presented  in 
solid  lines.  In  addition,  tic  marks  are  placed  on  the  circle  at  the  RMAX1  and  RMAX2 
locations.  Because  the  ends  of  the  double  thick  portion  of  the  circle  represent  RTR  and 
RMIN,  no  additional  tic  marks  are  required  for  these  parameters.  In  addition,  when 
target  range  is  less  than  RMIN,  a  conventional  “Break-X”  should  be  superimposed  over 
the  NIRD  symbology.  The  word  “SHOOT”  should  also  be  displayed  just  inside  the 
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upper  portion  of  the  NIRD/ASE  circle  whenever  weapon  launch  parameters  are  met, 
target  range  is  between  RMAX2  and  RMIN,  and  the  target  is  not  identified  as  Friendly, 
Neutral,  or  Unknown.  When  range  is  between  RMAX1  and  RMIN,  and  all  other  launch 
parameters  are  met,  the  SHOOT  cue  should  be  flashed  at  a  5-Hz  rate. 

The  recommended  implementation  of  the  NIRD  technique  “normalizes”  the  range 
performance  of  Own  Ship  missiles  so  that  specific  critical  regions  of  the  weapon’s 
performance  envelope  are  always  represented  by  the  same  quadrant  of  the  NIRD  circle, 
providing  ease  of  interpretation  and  enhanced  SA  regardless  of  radar  range  scale 
selection,  etc. 

Target  aspect  angle  is  represented  by  a  short  radial  line  (Aspect  Index)  fixed  to  the 
outside  of  the  NIRD/ASE  circle,  with  its  position  around  the  periphery  of  the  circle 
representing  the  target’s  nose  position  relative  to  the  LOS.  So  when  the  Aspect  Index  is 
located  at  12:00  on  the  circle,  target  aspect  is  zero  (i.e.,  tail-on),  3:00  represents  a  right 
90°  aspect  (right  side  of  target  visible),  6:00  represents  180°  aspect  (nose-on),  and  9:00 
indicates  a  left  90°  aspect.  Both  the  Aspect  Index  and  the  Range  Index  may  be  color 
coded  according  to  MIL-STD-1787C  guidance  for  Friendly,  Hostile,  Neutral,  or 
Unknown  when  this  intelligence  is  available  to  the  system.  Otherwise,  the  color  is 
standard  green. 

The  NIRD  symbology  should  appear  head-stabilized  in  the  HMD  whenever  a  full-system 
sensor  track  is  achieved  and  an  appropriate  weapon  is  available  and  selected.  When 
displayed,  the  NIRD  symbology  should  be  centered  at  the  same  position  on  the  HMD  as 
the  Aiming  Reference,  and  should  replace  that  symbology.  The  NIRD  symbology  should 
also  be  displayed  on  the  VHUD,  aircraft  stabilized,  when  that  display  appears  in  the 
HMD  FOV. 

The  target  locator  line  and  target  reference  symbol  associated  with  the  NIRD  should  be 
reflective,  as  described  above,  and  both  may  be  color  coded  according  to  target 
identification  as  with  the  Aspect  and  Range  Indexes.  The  reflective  cue  symbol  should 
correspond  to  the  guidance  provided  in  MIL-STD-1787C  for  threat  symbology,  with  the 
added  feature  that  its  alignment  relative  to  the  TLL  provides  a  redundant  target  aspect 
cue.  The  orientation  of  the  nose  of  the  threat  symbol  in  relation  to  the  center  of  the 
NIRD/ASE  circle  denotes  aspect  angle.  When  pointed  directly  along  the  TLL  toward  the 
center  of  the  circle,  target  aspect  is  180°  (nose-on),  etc.  Target  altitude  is  displayed 
digitally  immediately  to  the  right  of  the  target  symbol  in  the  Thousands-Hundreds  format 
(i.e.,  25-5  would  indicate  25,500  ft  target  altitude),  and  target  calibrated  airspeed  (to  the 
nearest  kt)  is  provided  digitally  to  the  left  of  the  target  symbol.  Target  type  is  displayed 
alphanumerically  above  the  target  symbol,  and  actual  target  range  to  the  nearest  0.1  NM 
is  displayed  immediately  below  the  target  symbol.  Whenever  a  TLL  is  displayed,  a 
digital  read-out  is  provided  near  the  center  of  the  NIRD  circle  opposite  to  the  orientation 
of  the  TLL,  indicating  the  number  of  degrees  from  NIRD  center  to  the  target  LOS.  These 
concepts  are  illustrated  in  Figure  16.  It  should  be  recognized  that  many  aspects  of  the 
NIRD  concept  are  relatively  new  and  innovative,  and  should  be  the  subject  of  further 
study. 
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It  should  be  recognized  that  it  may  be  possible  to  have  shot  opportunities  without  having 
the  sensor  (presumably  radar)  track  that  would  be  required  to  generate  Launch 
Acceptability  Region  (LAR)  symbology  such  as  the  NIRD.  An  example  of  this  would  be 
the  AIM-9  SIDEWINDER  class  of  air-to-air  missile.  In  that  case  it  is  still  left  to  the  pilot 
to  judge  acceptable  launch  parameters.  This  task  is  facilitated  by  standard  visual  and 
audio  missile  track  display  techniques  employed  on  current  fighters,  and  the  fact  that 
most  of  these  situations  occur  WVR.  The  NIRD  should  be  provided,  however,  any  time 
sufficient  data  exists  for  its  creation.  These  parameters  may  be  derived  from  Own-Ship 
sensors,  like  radar,  or  from  off-board  sources  via  datalink. 

Secondary  target  tracks  (i.e.,  not  designated  as  primary)  within  the  HMD  FOV  should  be 
displayed  at  the  pilot’s  option  (subject  to  declutter)  by  similar  symbology,  with  the 
addition  of  the  associated  target  priority  number  immediately  below  the  target  symbol. 
These  secondary  target  symbols  can  “peg”  at  the  edge  of  the  HMD  FOV  when  outside 
that  FOV,  but  would  not  have  associated  TLLs.  A  rapid  method  of  changing  target 
priorities  should  also  be  implemented.  This  might  allow  the  pilot  to  “Quick  Pick”  a 
secondary  target  by  simply  placing  the  HMD  Aiming  Reference  (either  the  Aiming  Cross 
or  the  NIRD  circle,  depending  on  circumstances)  on  that  target  and  commanding  a  shift 
to  Priority  1  status,  either  through  either  HOTAS  or  DVI.  All  other  tracked  targets  would 
then  shift  priority  automatically  according  to  their  original  relative  order. 
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Once  the  primary  designated  target  (PDT)  enters  the  HMD  FOV,  the  target  symbol 
should  be  replaced  by  a  Target  Designator  (TD)  Box.  The  shape  of  the  TD  Box  should 
be  unique  for  each  tracking  sensor.  For  instance,  square  for  radar,  circular  for 
SIDEWINDER,  hexagonal  for  infrared  search  and  track  (IRST),  etc.  Only  one  TD  Box 
should  be  displayed  for  each  sensor.  The  alphanumeric  information  for  the  target  would 
remain,  but  an  Aspect  Index  similar  to  the  one  described  affixed  to  the  NIRD/ASE  circle, 
may  be  similarly  attached  to  the  TD  Box  to  provide  target  aspect  angle  in  lieu  of  the 
reflective  TLL  symbology  when  that  information  is  available.  The  word  “SHOOT” 
should  appear  immediately  above  the  TD  Box  as  well  as  inside  the  upper  portion  of  the 
NIRD/ASE  circle  when  all  conditions  have  been  met,  as  listed  above.  This  symbology  is 
depicted  in  Figure  17. 

In  addition  to  the  above,  the  SMEs  in  this  study  recommended  several  unique  features  to 
enhance  SA  in  the  air-to-air  mission.  One  such  suggestion  is  an  “Off-Weapon  Cue,”  that 
would  alert  the  pilot  if  the  weapon  selected  is  not  optimal  for  the  current  situation.  For 
instance,  the  pilot  may  have  SIDEWINDER  selected  with  a  target  outside  maximum 
range  for  that  weapon,  but  well  within  range  of  AMRAAM,  which  may  also  be  available. 
In  such  a  case,  the  words  “OFF  WEAPON”  might  be  in  place  of  the  SHOOT  cue,  and 
flashed  at  a  5-Hz  rate.  An  addition  to  this  system  would  be  similar  to  the  OFF  WEAPON 
cue  currently  implemented  in  some  F-15s.  This  system  provides  the  pilot  a  cue  whenever 
AMRAAM  is  selected,  but  a  valid  shot  also  currently  exists  for  SIDEWINDER.  The  logic 
behind  this  approach  is  that  SIDEWINDER  is  much  less  expensive  than  AMRAAM,  so  it 
is  the  preferred  weapon  from  a  logistics  standpoint  when  within  the  LAR  of  both 
missiles.  Symbology  similar  to  that  currently  deployed  in  the  F-15  could  be  used  for  this 
function. 


SHOOT 

SU27 
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25-5 


FIGURE  17:  TD  BOX  SYMBOLOGY 

(Su-27  Target,  Within  Optimal  Firing  Parameters,  565  KCAS,  25,500  MSL,  10° 

Right  TAA,  24.2  NM  Range) 


Another  such  suggestion  is  called  a  “Clear-Avenue-of-Fire  (CAF)  Cue.  In  actuality  this 
would  be  an  “UNCLEAR”  avenue  of  fire  cue  that  would  alert  the  pilot  to  another  aircraft 
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projected  to  be  dangerously  close  to  the  predicted  path  of  the  weapon  selected  if  fired  at 
the  current  priority  target.  Providing  such  a  cue  could  be  technologically  challenging, 
since  the  aircraft  would  need  to  be  able  to  track  multiple  targets  in  a  high-resolution  3-D 
mode  (like  multiple  “single-target  tracks”)  to  provide  the  required  accuracy.  This 
capability  may,  however,  be  available  in  some  next-generation  fighters.  The  suggested 
format  for  this  display  is  a  red  “X”,  with  dimensions  about  twice  that  of  the  TD  Box, 
superimposed  over  the  TD  Box  and  inside  the  NIRD  circle,  and  the  letters  “CAF’  in  the 
positions  normally  reserved  for  the  SHOOT  cue  when  this  condition  is  detected. 

In  addition  to  the  head-stabilized  NIRD  symbology  on  the  HMD,  the  SMEs  in  this  study 
recommend  that  a  more  conventional  DLZ  symbology  be  included  on  the  VHUD. 
Although  there  was  no  objection  to  simply  adopting  the  current  F-15  ladder-format  DLZ 
symbology,  some  enhancements  are  discussed  on  the  following  section. 


6.22  THREAT  WEAPONS  ENVELOPE 

6.22.1  FUNCTIONALITY 

Successful  air-combat  missions  depend  on  defensive,  as  well  as  offensive  capability.  A 
critical  part  of  the  defensive  equation  is  pilot  SA  on  the  location,  capabilities,  and  status 
of  current  and  potential  threats.  Display  of  threat  location,  identification,  and  status  are 
discussed  above.  Particularly  when  engaging  modem  all-aspect  air-to-air  threats,  or 
operating  in  an  environment  of  active  ground  threats,  it  is  critical  that  the  pilot  have 
timely  assistance  in  assessing  the  weapons  capability  of  the  enemy. 

6.22.2  FORMAT  AND  MECHANIZATION 

When  sufficient  information  on  an  air-to-air  target  is  available  to  perform  the  necessary 
calculations,  a  “combination  DLZ,”  would  be  very  useful.  This  DLZ  would  provide  Own 
Ship  launch  parameters  as  usual,  but  also  present  the  calculated  launch  parameters  of  the 
enemy  aircraft.  A  suggested  format,  depicted  in  Figure  18,  would  be  a  double  vertical 
scale  with  the  scale  and  critical  weapon  range  tics  for  Own  Ship  on  the  right  side  colored 
green,  and  a  similar  vertical  scale  and  range  tics  on  the  left  colored  red.  Target  range 
would  be  displayed  in  the  center,  between  the  two  vertical  scales,  by  a  diamond-shaped 
“double  index”  read  against  range-scale  tic  marks  on  each  DLZ,  marked  in  NM.  Closure 
rate  may  be  provided  inside  the  range  index  symbol.  Such  a  display  would  provide  the 
pilot  with  much  improved  SA  regarding  how  close  Own  Ship  can  approach  the  target  to 
optimize  weapon  performance,  versus  the  level  of  threat  involved.  It  would  also  offer 
valuable  insight  into  the  most  appropriate  defensive  action  to  take  if  an  enemy  weapon  is 
detected. 
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Selected  Radar 
Range  Scale 


FIGURE  18:  DOUBLE  DYNAMIC  LAUNCH  ZONE  SYMBOLOGY 

Obviously,  such  a  display  would  require  a  significant  level  of  detail  on  the  enemy  aircraft 
type  and  probable  weapons.  Current  Electronic  Identification  (EID)  systems  can  often 
provide  aircraft  type,  but  probably  not  a  weapon  type,  at  least  not  until  after  it  is  actually 
fired.  In  this  case,  an  assumption  can  be  made  that  the  target  is  carrying  the  most 
effective  weapon  it  is  capably  of  employing,  to  be  on  the  safe  side. 

It  is  recognized  that  the  Priority  1  target,  for  which  the  threat  weapon  parameters  would 
be  displayed  as  suggested  above,  may  not  represent  the  greatest  current  threat  to  Own 
Ship.  Therefore,  it  would  also  be  very  valuable  to  provide  an  indication  of  the  LARs  of 
all  known  threats,  or  at  least  the  highest  threat.  Probably  the  most  appropriate  medium 
for  this  information  would  be  the  HDD,  in  association  with  an  audio/voice  cue. 

In  addition  to  the  air-to-air  threat  DLZ  described  above,  it  is  recommended  that  the 
option  be  provided  to  display  the  LAR  of  ground  threats.  The  recommended  format, 
illustrated  in  Figure  19,  for  presenting  this  information  on  the  HMD  and  VHUD  is  the 
“threat  bubble.”  The  threat  bubble  is  envisioned  as  a  3-D  earth-stabilized  “balloon” 
representation  of  the  threat  envelope,  centered  on  the  location  of  the  actual  threat.  The 
basis  of  the  threat  envelope  should  be  selectable  by  the  pilot  to  represent  either  maximum 
aerodynamic  range,  “doctrinal”  firing  range  (i.e.,  the  range  at  which  the  threat  might  be 
expected  to  fire  based  on  past  observations  or  enemy  doctrine),  maximum  effective  range 
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depending  on  current  position,  altitude,  speed,  course,  and  signature  of  Own  Ship,  etc. 
The  threat  bubbles  should  be  transparent  and  colored  red. 
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FIGURE  19:  SYNTHETIC  TERRAIN  WITH  THREAT  BOUNDARY 


When  Own  Ship  is  actually  located  inside  a  threat  bubble,  the  outline  of  the  horizontal 
limits  of  the  bubble  should  be  pictured  on  the  outside  view  of  the  ground  by  a  broken  red 
line  with  lighter  red  shading  (tinting)  over  the  entire  HMD  FOV.  Any  “black-line” 
course  displayed  on  the  ground  or  any  pathway-in-the-sky  gates  should  be  colored  red 
when  located  inside  the  threat  bubble.  In  addition,  a  “nearest  exit”  arrow  should  be 
displayed  on  the  HMD  or  VHUD  pointing  to  the  nearest  boundary  of  the  threat  bubble 
around  Own  Ship.  This  arrow  should  point  to  the  LOS  of  the  nearest  exit  point,  both 
horizontally  and  vertically,  and  include  the  number  of  degrees  from  the  current  HMD  to 
the  exit  point.  When  the  exit  point  is  actually  within  the  FOV  of  the  HMD,  the  arrow 
would  change  to  a  solid  circle  with  “fins,”  as  though  looking  down  the  shaft  of  the  arrow. 
The  exit  point  itself  (circle  with  fins)  should  be  earth  stabilized,  but  the  arrow  pointing  to 
it  is  head  stabilized.  Illustrations  of  this  symbology  are  depicted  in  Figures  20a  and  20b. 

It  should  be  noted  that  contrast  and  readability,  especially  with  multi-color  displays,  is 
critical,  and  virtually  impossible  to  predict  with  any  certainty  without  a  great  deal  of 
evaluation  with  the  actual  operational  hardware  and  software.  Therefore,  all  color 
recommendations  in  this  report  should  be  considered  as  suggested  guidelines  only. 
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FIGURE  20a:  INSIDE  THREAT  ZONE  WITH  EXIT  ARROW  DEPICTED 
(Nearest  Exit  Point  160°  Left  and  Slightly  Down,  Outside  HMD  FOV) 
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FIGURE  20b:  INSIDE  THREAT  ZONE  WITH  EXIT  ARROW  DEPICTED 
(Nearest  Exit  Point  Inside  HMD  FOV) 
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A  further  enhancement  to  ground  threat  weapons  capability  awareness  would  be  the 
implementation  of  a  DLZ  similar  to  that  described  above  for  air-to-air  threats.  Suggested 
mechanization  would  have  the  pilot  designate  the  threat  of  interest  by  placing  the  Aiming 
Reference  over  the  threat  icon,  then  commanding  the  threat  DLZ  by  either  HOT  AS  or 
DVI.  The  calculated  DLZ  for  the  threat  could  then  be  displayed  on  the  VHUD,  in  a 
manner  similar  to  the  left  side  of  the  DLZ  depicted  in  Figure  18. 


7.0  CONCLUSIONS 

The  CWA  process,  as  applied  in  this  study,  resulted  in  a  considerable  number  of 
innovative  approaches  and  recommendations  related  to  a  rather  formidable  task.  It 
should  be  understood,  however,  that  the  recommendations  provided  in  this  report  are 
largely  subjective,  and  that  many  issues  require  additional  study  to  verify  their 
effectiveness  and  usefulness.  The  number  of  SMEs  involved  in  this  study  was  limited, 
but  the  experience  level  was  both  diverse  and  extensive.  The  qualifications  of  these 
SMEs  should  tend  to  improve  the  validity  of  the  resulting  concepts  and 
recommendations,  since  their  experience  provides  a  basis  for  understanding  the  esoteric 
operational  implications  involved.  We  consider  highly  experienced  SMEs  to  be  crucial 
to  both  this  effort  and  CWA  in  general. 

It  should  also  be  understood  that  the  recommendations  provided  above  are  not 
“specifications”  in  the  strictest  sense,  because  they  were  obtained  through  subjective 
techniques  from  a  very  small  sample  of  subjects.  They  still  validation,  such  as  laboratory 
and/or  field  testing,  review  by  a  larger  pilot  base,  etc.  These  recommendations  do, 
however,  provide  a  rich  vase  for  further  study  and  evaluation. 
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APPENDIX  A 


ANALYSIS  OF  HMD  SYMBOLOGY  REQUIREMENTS  FOR  THE 
F-15E  NIGHT  MOBILE-TARGET  /  SCUD-HUNT  MISSION 


The  phases  of  the  F-15E  Mobile-Target  /  SCUD-Hunt  mission,  aside  from  the 
generic  phases  of  any  mission  (i.e.,  planning,  preflight,  taxi,  takeoff,  landing, 
debriefing,  etc.),  are  identified  as  follows: 


>  Ingress 

>  Target  Search  /  Detection  /  Identification 
^  Weapon  Delivery 

>  Egress 

Addressing  each  of  these  phases,  we  will  endeavor  to  establish  the  requirements  for 
HMD  symbology  to  support  the  essential  functions  inherent  in  the  tasks  involved. 

INGRESS 

A  night  ingress  into  hostile  territory  for  a  SCUD-Hunt  mission,  depending  on  the 
assessed  threat,  might  be  planned  for  either  medium-high  altitude,  or  for  low-level.  For  a 
medium-high  altitude  ingress,  only  typical  “navigation”  tasks  are  required,  in  addition  to 
tasks  implicit  in  any  combat  environment,  such  as  communicating  with  wingmen  and 
support  assets,  monitoring  for  threats,  etc.  For  night  low-level  ingress,  the  F-15E  will 
most  probably  employ  Forward-Looking  Infrared  (FLIR)  or  Night- Vision  Goggles 
(NVGs)  and  Terrain-Following  Radar  (TFR). 

Effective  navigation  in  a  combat  environment  requires  the  following  functions: 

•  Heading  /  Attitude  /  Altitude  /  Airspeed  Control  and  Monitoring 

•  Own  Aircraft  Status  (including  fuel  state)  Monitoring 

•  Location  Monitoring 

•  Waypoint  Timing 

•  Coordination  with  Wingmen  and  Supporting  Assets 

•  Threat  Monitoring  /  Evasion 

These  functions  are  required  for  all  phases  of  the  mission,  but  will  only  be  discussed  for 
the  Ingress  phase.  The  format  and  mechanization  of  the  HMD  for  these  functions  may 
vary  among  phases.  Each  of  these  functions  will  be  discussed  in  order. 

Heading,  /  Attitude  /  Altitude  /Airspeed  Control  and  Monitoring,  This  function  requires 
some  display  of  own-aircraft  heading  (at  least  magnetic,  with  course  also  desirable), 
attitude  (pitch  and  roll),  altitude  (at  least  barometric,  with  absolute  also  a  virtual 
requirement  for  low-level  ingress),  and  airspeed  (at  least  KCAS,  with  KTAS  and  Mach 
desirable).  Climb  /  Dive  Rate  and  Velocity  Vector  /  Flight  Path  Marker  are  also 
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desirable.  The  HMD  should  preferably  be  able  to  accommodate  display  of  these  data 
elements,  although  it  is  recognized  that  all  might  not  be  required,  or  desirable,  under  all 
circumstances.  Therefore,  effective  “declutter”  options  need  to  be  provided. 

To  facilitate  low-level  flight  and  navigation  at  night,  FLIR  imagery  on  the  HMD  is  highly 
desirable.  A  substitute  for  this  capability  might  be  the  use  of  NVGs,  so  the  HMD  should 
be  NVG  compatible. 

Consideration  should  also  be  given  to  providing  TFR  information  on  the  HMD.  The 
standard  TFR  “E-Scope”  display  is  probably  not  appropriate  for  the  HMD,  but  the 
information  currently  available  on  the  Head-Up  Display  (HUD)  should  be  provided.  This 
includes  TF  pitch  command  information,  radar  altitude,  and  associated  WARNING  and 
CAUTION  indications. 

Own  Aircraft  Status  (including  fuel  state )  Monitoring.  This  function  requires  some 
display  of  aircraft  system  status,  including  Aircraft  Master  Mode  (i.e.,  NAV,  A/A,  A/G), 
weapon  selection,  status  (or  at  least  alerting  to  abnormal  parameters)  of  all  aircraft 
systems  (i.e.,  engine,  electrical,  hydraulic,  flight  controls,  etc.),  and  fuel  state.  Although 
Aircraft  Master  Mode  might  be  obvious  due  to  other  display  characteristics,  it  is  probably 
prudent  to  provide  an  explicit  indication  of  Mode  on  the  HMD,  especially  since  this  is  a 
relatively  minor  requirement.  Similarly,  weapon  selection/status  (i.e.,  Armed/Safe)  will 
most  probably  be  provided  on  a  head-down  display  (HDD),  but  inclusion  on  the  HMD  is 
probably  desirable  also,  with  minimal  impact  (it  is  desired  that  inclusion  of  weapon 
selection/status  have  minimal  impact?  Or  it  will  have  minimal  impact?  How  do  we 
know?).  Aircraft  system  status,  on  the  other  hand,  generally  does  not  require  constant  or 
instant  attention,  except  in  the  case  of  abnormalities.  Therefore,  it  should  be  adequate 
simply  to  provide  for  an  alerting  function  for  the  HMD,  such  as  a  flashing  MASTER 
CAUTION  or  WARNING  indicator  when  required,  to  alert  the  pilot  to  attend  to  other 
displays.  Similarly,  fuel  state  does  not  normally  require  constant  attention,  but  periodic 
attention  is  called  for,  as  well  as  alerting  to  critical  conditions,  such  as  JOKER/BINGO 
fuel  state.  The  latter  function  can  be  handled  in  the  same  manner  as  system 
abnormalities.  To  maximize  head-out  time  for  other  functions,  and  to  reduce  pilot 
workload,  however,  it  would  be  helpful  to  provide  an  indication  of  projected  fuel  state  at 
the  selected  waypoint  for  planning  purposes.  Again,  effective  declutter  options  are  a 
requirement. 

Location  Monitoring.  This  is  the  typical  “navigation”  function  of  locating  own-ship 
within  the  area  of  interest.  This  function  is  normally  provided  by  an  HDD,  which  might 
take  the  form  of  a  moving  map,  and  usually  is  presented  as  a  “God’s-Eye  View.” 
Although  the  usual  display  technique  has  been  proven  to  be  effective  for  “global” 
navigation  purposes,  and  such  displays  are  generally  not  appropriate  for  HMDs,  it  may  be 
of  some  value  to  provide  “local”  navigation  assistance  on  the  HMD.  At  a  minimum,  this 
would  probably  include  display  of  the  selected  navigation  waypoint,  stabilized  and 
conformal  with  the  terrain.  Other  possibilities  to  reduce  pilot  workload  include 
assistance  in  maintaining  a  direct  track  to  the  selected  waypoint.  This  could  take  the 
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form  of  a  heading  caret,  course-deviation  indicator  (CDI),  flight  director,  “pathway-in- 
the-sky,”  etc. 

Waypoint  Timing.  A  typical  requirement  of  any  A/G  combat  mission  is  “Time  on 
Target,”  (TOT).  Although  this  requirement  may  be  somewhat  less  restrictive  for  the 
selected  mission  of  interest  (Mobile-Target  /  SCUD-Hunt),  some  means  of  facilitating 
this  ubiquitous  function  should  be  provided.  On  the  HMD  this  might  take  the  form  of  a 
“time-to-go  to  selected  waypoint,”  “estimated  time  over  selected  waypoint,”  “estimated 
time  early/late  (relative  to  planned  time)  over  selected  waypoint,”  “calculated  speed 
required  to  reach  the  selected  waypoint  at  the  planned  time,”  etc. 

Coordination  with  Wingmen  and  Supporting  Assets.  The  ability  to  communicate  and 
coordinate  is  assumed  for  any  flight,  whether  or  not  these  functions  are  actually 
exercised.  For  combat  missions,  a  reduction  in  communications  is  preferable,  while  not 
sacrificing  situation  awareness  (SA),  mission  effectiveness,  or  safety.  Typically, 
communications  may  be  required  to  monitor  the  positions  or  status  of  friendly  forces,  for 
station-keeping/formation  control,  to  coordinate  offensive/defensive  actions,  and  for 
mission  control  functions.  Voice  radio  and  visual  signals  are  the  traditional  means  of 
communication  among  military  aircraft,  but  visual  signals  may  be  limited  during 
night/all-weather  combat  missions,  and  radio  transmissions  may  be  discouraged  because 
of  the  danger  of  detection  and  exploitation  by  the  enemy.  FLIR  imagery  on  the  HMD, 
particularly  in  conjunction  with  an  all-aspect  head-steered  FLIR  system,  would  be 
advantageous  in  facilitating  visual  signaling,  as  well  as  reducing  the  need  for  some 
communications.  Therefore,  FLIR  imagery  on  the  HMD  is  considered  highly  desirable 
for  this  function,  as  well  as  for  other  purposes  during  this  mission.  Likewise,  datalink 
(LINK- 16)  may  be  preferable  to  voice  radio  in  providing  communications  for  many 
position,  status,  coordination,  and  mission  control  functions.  Symbolic  display  of  datalink 
information  on  the  HMD,  such  as  nearby  aircraft  location,  threat  positions  (and  possibly 
status,  threat  ranges,  etc.),  target  location,  etc.,  is  also  promising.  3-D  Audio  capability 
may  also  contribute  to  these  functions. 

'Threat  Monitoring.  /  Evasion.  The  inclusion  of  FLIR  imagery  and  datalink  symbology 
on  the  HMD,  as  described  above,  is  also  of  considerable  value  for  this  function.  In 
addition,  own-ship  Radar  Warning  Receiver  (RWR)  or  Threat  Warning  System  (TWS) 
information  may  be  critical.  This  display  could  take  the  form  of  an  indication  of  threat 
azimuth  or  line-of-sight  (LOS)  as  available,  to  aid  the  pilot  in  visually  locating  and/or 
avoiding  the  threat.  This  also  appears  to  be  a  promising  application  for  3-D  Audio 
technology.  If  automated  threat  reactions  are  anticipated,  such  as  expendables  (chaff, 
flares,  decoys,  etc.)  integrated  with  a  TWS,  or  automated  active  electronic 
countermeasures  (ECM)  functions,  their  operating  status  would  also  be  a  valuable 
addition  to  the  HMD  so  that  the  pilot  can  remain  head  out  of  the  cockpit  during  threat 
evasion  maneuvering.  In  fact,  some  indication  of  defensive  system  function  is  desirable 
whether  automated  or  manually  deployed.  This  display  might  take  the  form  of  a  simple 
CHAFF,  FLARE,  DECOY,  or  ECM  indication  when  the  named  system  is  activated. 
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Threats,  ground  and  air,  may  also  be  displayed  and  monitored  via  datalink  (i.e.,  LINK-16 
/  JTEDS).  The  HMD  should  optimally  be  able  to  display  datalink  targets.  In  addition, 
own-aircraft  Air  Intercept  (AI)  radar  is  a  vital  source  for  airborne  threats.  Although  a 
complete  AI  radar  picture  is  probably  best  suited  for  a  head-down  God’s  Eye  display 
format,  any  target  designated  as  the  Primary  Designated  Target  (PDT)  or  as  a  Single- 
Target  Track  (STT)  should  be  displayed  on  the  HMD  for  rapid  pilot  visual  acquisition 
and/or  targeting.  There  should  be  some  form  of  Target  Locator  Line  (TLL)  symbology  to 
indicate  target  azimuth  and  elevation  relative  to  the  HMD  LOS  (and/or  possibly  the 
aircraft  longitudinal  axis)  whenever  the  target  lies  outside  the  current  HMD  field-of-view 
(FOV).  It  may  also  be  useful,  under  some  circumstances,  to  display  additional  targets 
being  tracked  by  the  AI  radar  at  a  reduced  status.  Any  AI  radar  targets  displayed  on  the 
HMD  should  be  clearly  differentiated  from  displayed  datalink  targets  in  some  manner. 

AI  radar  Auto  Acquisition  modes  must  be  supported  by  the  HMD.  Air-to-air  weapon 
symbology  should  also  be  accommodated  by  the  HMD,  and  could  be  similar  to  current 
HUD  functionality. 


TARGET  SEARCH  /  DETECTION  /  IDENTIFICATION 

Target  Search.  This  function  may  be  conducted  visually,  with  the  support  of  FLIR 
and/or  NVGs,  by  A/G  radar,  or  with  the  aid  of  off-board  sources  via  datalink.  As 
discussed  above,  A/G  radar  is  probably  not  an  optimal  sensor  for  HMD  applications.  The 
display  of  FLIR  imagery  and  datalink  targets  on  the  HMD,  as  provided  above,  should 
adequately  support  this  function.  Consideration  should,  however,  be  given  to  providing 
for  inset  Synthetic-Aperture  Radar  (SAR)  imagery  on  the  HMD. 

Target  Detection.  The  comments  regarding  Target  Search  also  apply  to  this  function.  In 
addition,  some  sort  of  Ground  Moving  Target  Indicator  (GMTI)  system  maybe  available, 
so  symbology  appropriate  for  such  a  system  should  be  provided  on  the  HMD  to  assist  in 
discriminating  possible  targets  from  background  terrain. 

Target  Identification  (ID}.  In  general,  targets  may  be  identified  either  visually  or 
electronically.  Assistance  may  also  be  provided  by  off-board  sources,  either  via  datalink, 
marking,  or  voice  communications.  Visual  ID  may  be  facilitated  by  image 
magnification.  Therefore,  a  magnified  FLIR  image  is  a  candidate  for  HMD  application. 
The  nonconformal  nature  of  such  a  display,  however,  would  probably  indicate  that  such  a 
feature,  if  included  on  the  HMD,  should  be  implemented  as  an  inset,  rather  than  replacing 
the  conformal  imagery  FLIR.  An  alternate  approach  would  be  to  confine  magnified 
imagery  to  an  HDD,  controlled  by  the  HMD  LOS. 

Electronic  target  ID  may  be  provided  by  either  own-aircraft  or  off-board  systems.  It 
could  take  the  form  of  either  graphical  or  alphanumeric  symbology.  The  HMD  is  a 
candidate  for  additional  symbology  to  facilitate  this  function.  Alphanumeric  and/or 
distinct  graphical  symbology  should  be  considered.  The  source  of  this  ID  should  also  be 
clearly  apparent  in  any  display.  Targets  may  also  be  identified  through  “marking”  by  an 
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off-board  source  (i.e.,  ground  observer,  RPV,  etc.),  normally  by  laser  designator. 
Therefore,  some  means  of  displaying  such  a  designation  should  be  provided  on  the  HMD. 


WEAPON  DELIVERY 

Weapon  delivery  tasks  begin  at  the  point  of  target  ID,  and  end  with  either  the  launch  or 
impact  of  the  weapon,  depending  on  weapon  characteristics.  These  tasks  include: 

•  Weapon  Selection 

•  Attack  Planning 

•  Attack  Execution 

•  Target  Acquisition  /  Designation  /  Lock 

•  Weapon  Launch 

•  Weapon  Support 

Weapon  Selection.  The  Weapon  Selection  task  is  discussed  above  under  the  Ingress 
phase  [Own  Aircraft  Status  (including  fuel  state)  Monitoring]. 

Attack  Planning.  This  is  a  complex  task  that  requires  considerable  preflight 
consideration  and  preparation.  The  Mobile  Target  /  SCUD-Hunt  mission  places 
additional  significant  burdens  on  the  pilot,  by  virtue  of  time  limitations,  a  high-workload 
environment,  and  the  integration  of  real-time  information  into  the  attack  plan.  Other  than 
providing  enhanced  SA  and  reducing  pilot  workload,  it  is  unlikely  that  an  HMD  can 
contribute  directly  to  this  task. 

Attack  Execution.  This  task  involves  maneuvering  to  achieve  weapon  launch  parameters 
in  the  most  effective  and  expeditious  manner,  while  minimizing  threat  exposure. 

Subtasks  include  navigation,  aircraft  control,  and  threat  detection/avoidance.  These 
subtasks  have  all  been  discussed  to  some  degree  above  in  relation  to  the  Ingress  phase  of 
the  mission.  Due  to  the  likelihood  of  increased  workload,  divided  attention,  and  severity 
of  maneuver  dynamics  during  this  phase,  as  opposed  to  the  Ingress  phase,  many  of  the 
“nice-to-have”  features  discussed  above  for  the  HMD  become  more  critical,  or  even 
essential.  Monitoring  of  the  aircraft’s  state  relative  to  the  terrain,  including  attitude, 
altitude  (especially  absolute  altitude),  and  location  relative  to  the  target,  threats,  and  other 
relevant  waypoints  becomes  more  critical.  HMD  symbology  must  be  highly  intuitive  to 
support  these  functions  in  the  Attack  Execution  environment,  so  such  features  as  TLLs 
and  flight-path  guidance  take  on  added  importance. 

Threat  monitoring  and  evasion  continue  to  be  concerns  during  the  Attack  Execution 
phase,  as  in  all  other  phases  of  the  mission.  This  task  was  discussed  above  under  the 
Ingress  phase  [Threat  Monitoring  /  Evasion],  Due  to  the  demands  of  the  Attack 
Execution  task,  this  subtask  becomes  more  challenging.  In  addition  to  the  associated 
increase  in  workload,  dynamic  maneuvering,  etc.,  it  is  likely  that  tighter  restrictions  will 
be  placed  on  navigation  and  maneuver  options  in  this  phase.  Virtually  all  weapons  have 
stringent  parameters  that  must  be  met  for  a  successful  launch,  and  satisfying  these 
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parameters  may  require  reduced  margin  of  error  in  avoiding  threat  launch  envelopes. 
Therefore,  this  subtask  demands  additional  attention  to  threat  avoidance/evasion 
symbology.  Specifically,  an  explicit  indication  of  threat  launch  envelopes  would  be  of 
great  value.  These  should  be  dynamic,  if  at  all  possible,  adjusting  to  the  current  speed, 
altitude,  and  maneuver  of  own  aircraft.  Although  graphical  threat-envelope  symbology 
should  probably  be  provided  on  an  HDD,  the  HMD  is  also  a  candidate  for  this 
symbology. 

Target  Acquisition  /  Designation  /  Lock.  Obviously,  this  task  is  highly  dependent  on  the 
weapon  to  be  employed.  The  weapons  currently  being  considered  for  this  mission  are: 

•  Iron  Bombs  /  Cluster  Bomb  Units  (CBUs) 

•  Laser  Guided  Bombs  (LGBs) 

•  MAVERICK  (AGM-65D/G) 

•  Joint  Direct  Attack  Munition  (JDAM) 

Iron  bombs  and  CBUs  are  normally  delivered  visually  with  the  aid  of  the  aircraft’s 
weapon  system.  Typical  delivery  modes  include  Continuously  Displayed  Impact  Point 
(CDIP),  for  which  the  weapon  system  calculates  and  displays  the  point  on  the  ground  at 
which  the  weapon  would  impact  if  released  at  a  given  instant.  Another  possibility  is  Dive 
Toss  mode,  for  which  the  pilot  designates  the  target  on  the  ground  and  the  weapon 
system  provides  steering  guidance  to  a  release  point.  The  delivery  aircraft  often  pulls  up 
into  a  climb  prior  to  release  to  increase  toss  range,  and  the  weapon  is  released 
automatically  at  the  proper  moment  to  reach  target  range.  A  third  possibility  is  a  Loft. 
This  mode  is  similar  to  Dive  Toss  mode,  except  that  the  target  is  more  likely  to  be 
designated  by  the  LANTERN  Targeting  Pod  (TGP)  or  SAR  rather  than  visually.  The 
TGP  may  also  be  used  in  a  more  direct-attack  mode  to  designate  a  target  for  a  system 
delivery,  similar  to  Dive  Toss 

To  support  the  visual  deliveries,  the  HMD  would  need  the  same  functionality  as  the  HUD 
now  provides  for  the  corresponding  delivery  mode,  and  could  furnish  identical 
symbology.  For  CDIP  that  includes  a  Displayed  Impact  Line.  For  Dive  Toss  or  Loft 
there  is  a  slewable  Target  Designator  (TD)  Diamond,  stabilized  on  the  ground.  In  the 
case  of  the  HMD,  the  TD  Diamond  might  be  mechanized  to  be  placed  near  the  target  by 
HMD  designation,  then  its  position  refined  by  Hands  On  Throttle  and  Stick  (HOT AS) 
control,  as  is  the  current  practice.  Additional  symbology  includes  an  Azimuth  steering 
Line,  Elevation  Steering  Line,  Target  Range,  etc. 

For  radar  Loft  the  target  is  designated  on  the  radar  display.  Although  this  is  normally  a 
head-down  function,  it  might  be  feasible  to  provide  an  inset  SAR  display  on  the  HMD  for 
this  purpose.  Once  the  target  is  designated,  the  position  of  the  TD  Diamond  should  be 
indicated  conformally  on  the  HMD  for  pilot  orientation.  The  associated  steering  bars, 
etc.,  could  be  displayed  non-conformally  on  the  HMD,  as  currently  on  the  HUD. 

LGBs  are  typically  delivered  in  a  manner  similar  to  that  described  above  for  the  Loft 
mode.  Target  designation  is  normally  performed  by  use  of  TGP  video,  either  directly  or 
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through  handoff  from  the  A/G  radar.  Following  weapon  release,  the  standard  procedure  is 
for  the  delivery  aircraft  to  turn  away  from  the  target  (to  the  right  for  the  F-15E)  to  avoid 
target  overflight  (maintain  standoff  distance),  and  designate  the  target  with  a  laser  spot 
from  the  TGP  during  the  final  seconds  of  the  weapon’s  flight.  The  location  of  the  laser 
spot  may  be  refined  until  weapon  impact.  Again,  this  is  normally  a  head-down  operation 
performed  by  the  Weapon  Systems  Officer  (WSO)  in  the  rear  cockpit  of  the  F-15E,  but  it 
may  be  feasible  to  provide  an  inset  box  of  TGP  video  on  the  HMD  to  support  this  task. 

Imaging  Infrared  (IIR)  MAVERICK  missiles  may  be  locked  onto  the  target  either  by 
slewing  the  MAVERICK  sensor  manually  while  observing  the  MAVERICK  IIR  video, 
or  by  handoff  from  the  TGP.  The  MAVERICK  seeker  might  be  initially  ground 
stabilized  near  the  target  by  using  the  HMD,  as  is  now  often  done  with  the  HUD,  then 
refined  by  slewing.  As  with  SAR  and  TGP  video,  an  inset  on  the  HMD  might  also  be 
provided  for  MAVERICK  video.  Additional  symbology  used  for  MAVERICK  includes 
an  indication  of  seeker  head  angle  prior  to  launch. 

Although  JDAM  is  normally  associated  with  “fixed”  targets,  for  which  pre-mission 
geographic  coordinates  are  available,  this  weapon  also  has  application  to  the 
Mobile-Target  /  SCUD-Hunt  mission  with  real-time  targeting.  Assuming  the  target 
is  stationary,  it  can  be  located  visually  or  by  FLIR,  and  its  coordinates  fixed  by  the 
TGP.  Coordinates  may  then  be  handed  to  the  JDAM  for  an  attack.  JDAM  delivery 
can  be  performed  by  the  various  modes  described  above  for  Iron  Bombs  /  CBUs,  or 
some  “Launch  Acceptability  Region”  (LAR)  symbology  might  be  provided  on  the 
HMD  to  permit  release  within  an  acceptable  dynamic  envelope. 

Weapon  Launch.  Minimal  support  is  required  from  the  HMD  for  this  subtask.  Simply 
removing  target  designation  and  weapon  aiming  symbology  from  the  HMD,  as  is 
normally  done  with  the  HUD,  should  suffice  to  indicate  weapon  launch. 

Weapon  Support.  No  support  is  required  for  Iron  Bombs  /  CBUs,  HR  MAVERICK,  or 
JDAM  after  launch.  LGBs,  however,  require  continuous  monitoring  and  refining  of  the 
laser  spot  position  till  weapon  impact.  This  may  be  accomplished  either  by  the  delivery 
aircraft,  another  supporting  aircraft,  or  a  ground  observer.  If  performed  by  an  aircraft 
equipped  with  TGP,  there  are  scan  and  masking  limits  that  must  be  observed  prior  to 
weapon  impact  in  order  that  the  laser  spot  remains  on  the  target.  The  TGP  scan  limits  are 
nominally  150  in  azimuth  and  down.  TGP  /  Laser  Masking  limits  are  more  restrictive 
and  aircraft  configuration  dependent.  These  limits  could  be  provided  in  some  manner  on 
the  HMD. 


EGRESS 

Egress  from  the  target  area  after  weapon  release  depends  largely  on  the  perceived  threat 
level.  Often  in  high-threat  scenarios  this  maneuver  is  performed  at  high  speed  and  low 
level.  The  considerations  discussed  above  for  the  Ingress  and  Weapon  Delivery  [Attack 
Execution]  phases  also  apply  to  this  phase. 
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SUMMARY 


In  summary,  the  table  below  lists  the  functions  required  by  an  HMD  if  it  is  to  support  the 
F-15E  Mobile-Target  /  SCUD-Hunt  mission  in  the  absence  of  a  HUD.  These  are 
categorized  as  either  “Required,”  or  “Desired,”  by  criticality,  but  for  all  practical 
purposes,  those  listed  in  the  Desired  column  are  also  quite  important  if  the  capability 
provided  by  the  current  HUD  is  to  be  maintained  or  improved.  The  order  of  listing  in 
each  group  is  random,  and  does  not  imply  any  relative  importance.  It  should  be  noted 
that  all  the  functions  listed  here  do  not  necessarily  need  to  be  provided  simultaneously. 
Functionality  required  is  largely  dependent  on  the  selection  of  Aircraft  Master  Mode  and 
system  selection.  Conventions  now  commonly  employed  for  modifying  HUD  display 
functions  and  formats  can  also  be  transferred  to  an  HMD.  Consideration  should  also  be 
given  to  declutter  options  to  allow  for  the  elimination  of  unnecessary  information  from 
the  HMD  when  appropriate.  Note  that  this  list  does  not  include  consideration  for 
supporting  tasks  not  specifically  associated  with  the  selected  mission,  including 
Instrument  Approaches,  etc. 

HMD  Functionality  for  Support  of  the 
F-15E  Mobile-Target  /  SCUD-Hunt  Mission 


DESIRED 

Course 

Attitude  (Pitch/Roll) 

Airspeed  (KTAS,  Mach) 

Altitude  (Baro/Radar) 

Velocity  Vector  /  FPM 

Airspeed  (KCAS) 

Climb  /  Dive  Rate 

Master  Caution  /  Warning 

TFR  Pitch  Command 

Fuel  State  Warning 

TFR  Warnings  /  Cautions 

Selected  WP  /  Target  Location 

Aircraft  Master  Mode 

Time-To-WP  Data 

Weapon  Selection  /  Status 

Ballistic  Weapon  Delivery 

Fuel  State  at  Selected  WP 

AI  Radar  Auto  Acquisition 

Course  Correction  Guidance 

FLIR  Imagery 

RWR/TWS  Warnings 

Expendables  / 

ECM  Employment 

AI  Radar  Target  Location 

A/A  Weapon  Launch  Parameters 

GMTI 

TGP  /  MAVERICK  /  SAR  Imagery  Inset 

Target  ID  /Designation 

Threat  Envelope 

TGP  /  Laser  Masking  Limits 
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APPENDIX  B 


CWA  SME  DESIGN  GROUP  MEETING  TOPICS 


Introduction:  We  are  trying  to  find  out  as  much  as  possible  about  the  information 
requirements  for  Air  to  Ground  Night  Strike  missions.  The  reason  we  are  asking  these 
questions  is  to  provide  inputs  to  the  design  of  an  advanced  Helmet  Mounted  Display  from 
a  Human-Centered  Design  perspective. 


1.  What  are  some  higher-level  guidelines  or  overriding  constraints  on  accomplishing 
mission  goals?  For  example,  we  are  thinking  about  things  like  Rules  of 
Engagement,  tactical  doctrine,  or  other  similar  things.  Can  you  describe  the  types 
of  things  contained  in  these  guidelines?  How  are  these  things  (all  things 
mentioned  in  the  questions)  typically  accessed  -  are  they  communicated  from 
somewhere,  or  kept  on  board  as  paper  or  computerized  documents,  or  pre¬ 
programmed  into  on-board  systems,  or  remembered  and  followed?  Are  these 
different  guidelines  always  in  agreement  -  if  not,  are  there  priorities  over  what 
should  be  followed  next? 

2.  Can  you  tell  us  about  the  different  kinds  of  systems  on  and  off  the  airplane  (e.g., 
sensors)  that  are  used  in  accomplishing  mission  tasks  and  how  they  interact? 
(Especially  coordination  between  on-board  and  off-board  systems).  Is  the  origin 
of  information  always  clear?  Is  the  certainty  of  information  always  clear? 

What  if  the  systems  we  just  talked  about  were  disabled  in  some  way  —  how  would 
the  ability  to  do  the  Mission  be  affected?  Are  there  some  systems  or  parts  of 
systems  that  could  be  used  as  substitutes  for  others  in  some  way? 

3.  What  are  the  decisions  that  are  typically  made  in  performing  night  mission 
functions?  What  are  the  tasks  that  have  to  get  done  to  accomplish  the  functions? 

4.  What  information  is  most  important  to  successful  mission  performance? 

5.  How  does  the  criticality  of  these  information  sources  vary  with  mission  phase? 
Ingress/Enroute,  N.O.T.E.,  IP,  Target,  Egress. 

6.  What  were  the  displays  like  that  support  mission  tasks?  Are  displays  well  done? 
Are  there  instances  in  which  your  information  requirements  were  not  satisfied  by 
current  displays? 

7.  Who  do  you  interact  with  during  mission? 

8.  What  are  the  communication  requirements  (within  element,  flight,  force)? 


78 


9.  Who  do  you  get  guidance  from?  What  type  of  guidance  did  you  have? 

10.  What  was  the  workload  rhythm  (peak  vs.  non-peak  periods)? 

11.  What  are  some  of  the  factors  that  complicate  night  strike  mission  performance  or 
make  a  mission  more  challenging?  Can  you  think  of  any  cases  where  this 
occurred?  Are  there  cases  in  which  difficult  tasks  could  be  made  easier  through 
better  information  display? 

12.  What  are  the  kinds  of  things  that  automation  or  better  displays  could  do  to  make 
the  mission  tasks  easier,  reduce  workload,  and  enhance  lethality,  survivability? 
(Note  -  assess  technical  feasibility  later). 

13.  Are  there  mission  profiles  that  are  not  used  during  night  operations  (pop  up)?  Are 
these  due  to  lack  of  visibility  or  the  target,  wingmen,  flights,  etc.? 

14.  What  enhancements,  if  any,  could  be  afforded  by  localized  audio? 

a.  Self  Defense  Systems 

b.  Nav/GCAS/TCAS 

c.  Keeping  track  of  wingmen,  flight,  etc. 

15.  What  enhancements,  if  any,  could  be  afforded  by  direct  voice  inputs? 

16.  How  could  various  advanced  HMD  technologies  effect  night  mission  tactics: 

a.  Night  vision  capability  affords  direct  visual  contact  with 

ground/friendlies/etc. 

b.  Data  link  enhanced  SORT,  target  /  shooter  pairing 
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